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ABSTRACT 

Terrain  navigation  is  a  critical  skill  in  the  military.  Virtual  environments  (VEs) 
have  been  suggested  as  a  possible  tool  in  training  spatial  knowledge.  However,  little 
research  has  been  conducted  into  the  ability  of  VEs  to  impart  spatial  knowledge  of  a  real 
world  area. 

This  thesis  research  addresses  the  utility  of  VEs  to  impart  spatial  knowledge  of  a 
natural  terrain  area  compared  to  traditional  methods.  Twenty  subjects  were  divided  into 
four  training  conditions  in  two  experiments.  The  first  experiment  had  a  VE  and  map-only 
group  and  trained  to  a  set  standard  rather  than  to  a  time.  The  second  experiment  also  had 
a  map-only  and  VE  group,  but  trained  one  hour  with  a  low  fidelity  map  (1:24,000  scale  as 
compared  to  1:5,000  scale  in  earlier  experiments).  Measures  were  taken  of  landmark, 
route,  and  survey  knowledge. 

The  results  suggest  that,  (1)  subjects  who  trained-to-standard  using  a  VE 
demonstrated  superior  route  and  landmark  knowledge  to  any  other  group,  (2)  spatial 
ability  plays  a  significant  role  in  navigation  performance,  and  (3)  adjusting  the  fidelity  of 
the  map  causes  individuals  to  adjust  their  planned  routes  to  the  information  that  is 
provided.  Furthermore,  while  good-map  reading  does  not  guarantee  success,  poor  map 
reading  skills  invite  failure.  Finally,  if  time  is  limited,  a  detailed  map  is  preferable  to 
other  methods. 
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I.  INTRODUCTION 


A.  PROBLEM  STATEMENT 

This  thesis  will  attenpt  to  answer  a  simple  question:  What  is  the  value  added  (if 
any)  of  using  a  virtual  enviroiraient  (VE)  as  an  augmentation  to  map  study  for  terrain 
familiarization.  The  thesis  will  analyze  and  compare  an  individual’s  acquisition  of  spatial 
knowledge  from  a  high-fidelity  virtual  representation  of  a  specific  area  to  those  trained 
with  traditional  map  techniques  (See  Chapter  El  Training).  The  research  intends  to 
investigate  unexplained  questions  from  earlier  research  and  to  study  the  validity  of  using 
virtual  environments  to  acquire  spatial  knowledge  of  an  area  of  real  world  terrain.  If 
proved  valid,  future  research  could  investigate  the  optimal  level  of  fidelity,  optimal  level  of 
immersion,  and  optimal  interface  control  devices  maximizing  knowledge  acquisition.  If 
disproved,  efforts  in  thi^  area  would  be  better  focused  on  other  types  of  tasks. 


B.  MOTIVATION 

1.  Anny  and  DOD  Relevance 

The  Department  of  Defense  (DOD)  is  actively  looking  for  ways  to  train  personnel 
while  conserving  precious  dollars  and  in5)roving  safety  and  performance.  Many  of  DOD’s 
explorations  have  involved  Conputer  Based  Trainers  (CBTs).  Virtual  environment 
trainers  and  simulators  have  proven  successful  for  combat  aircraft  simulators  [ANGI 
93][CRAN  93]  and  armored  vehicle  crews  [BOLD  85][BROW  88].  They  may  also  prove 
a  useful  technique  for  dismounted  soldiers  for  both  training  and  mission  rehearsal.  Such  a 
system  is  hoped  to  give  soldiers  performing  a  mission  better  situational  awareness. 

General  William  Hartzog,  commander  of  Training  Command  described  situational 
awareness  on  the  battlefield  to  be:  Knowing  where  you  are,  knowing  the  location  of 
friendly  forces,  and  knowing  the  location  of  enemy  forces.  Critical  to  knowing  where  you 
are  is  knowledge  of  the  ground  around  you  [BANK  97].  This  mirrors  the  advice  of  Sun 
Tzu  some  two  thousand  years  before:  “We  are  not  fit  to  lead  an  Army  on  the  march  unless 
we  are  familiar  with  the  face  of  the  country  -  it’s  mountains  and  forests,  its  pitfalls  and 
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precipices,  its  marshes  and  swamps  [SUN  83].”  Any  system  that  ailows  soldiers  to  better 
know  an  area  before  walking  the  ground  for  the  first  time  would  greatly  influence 
operations.  The  Army’s  doctrine  states  that  terrain  is  clearly  not  neutral  in  any  conflict. 
The  terrain  provides  an  advantage  to  the  side  that  better  understands  the  features, 
advantages,  and  limitations  of  a  given  area  and  plans  accordingly  [FMIO  93].  But  as  the 
American  Army  changes  firom  a  forward  deployed  force  to  a  power  projection  one  [CJCS 
98],  soldiers  and  leaders  are  aflbrded  less  opportunity  to  train  or  maneuver  on  the  ground 
they  may  later  have  to  fight  over.  By  virtue  of  the  enhanced  movement  ability  in  con5)uter 
environments,  individuals  may  be  able  to  explore  more  of  an  area  in  a  VE  than  if  actually 
walking  the  ground.  This  would  allow  soldiers  to  train  faster  than  before  in  some  cases 
through  time-con^ressed  training.  It  may  also  allow  them  to  become  familiar  with  an  area 
before  actually  arriving  there  in  person.  A  VE  would  also  allow  users  to  train  in  areas  that 
may  not  be  accessible  due  to  hostile  forces  or  prohibitive  travel  costs.  This  is  not  to 
suppose  that  VEs  could  entirely  replace  real  world  ground  training.  No  VE  in  the  near 
future  can  replicate  desert  training  down  to  the  sand  in  your  boots.  The  best  VEs  and 
simulators  augment  real  world  training,  they  do  not  replace  it  [SCHW  86]  [GATE 
87][BROW88]. 

One  foundation  of  mihtary  forces  is  conducting  rehearsals  prior  to  performing  a 
mission.  The  milkary  has  a  long  history  of  creating  mockups  as  part  of  these  rehearsals 
[GUN  95]  [MCRA  95]  [FINN  97]  [AMER  98].  These  aids  may  be  as  single  as  some 
circles  drawn  in  the  dirt  to  elaborate  buildings  and  structures  built  for  some  missions.  The 
one  thing  the  mihtary  has  not  been  able  to  do  is  make  the  terrain  at  the  rehearsal  site 
accurately  match  that  found  at  the  mission  site.  VEs  may  provide  a  cost-effective  way  of 
doing  just  that.  While  they  would  not  replace  the  need  for  actual  building  models,  they  can 
allow  soldiers  to  better  prepare  for  the  ground  around  target  areas.  As  a  mission  rehearsal 
tool,  they  would  allow  soldiers  to  rehearse  their  movement  to  and  fi'om  the  target  area, 
and  perhaps  explore  alternative  routes  to  ensure  the  optimal  selection. 

However,  just  because  a  training  tool  involves  a  con5)uter,  this  does  not 
automatically  make  in  better  than  alternatives  [HEAR  83].  Unlike  the  Army’s  Simulation 
Networking  (SIMNET)  program  [SCHW  86][BROW  88]  [BURN  90]  [ANGI  93],  there 
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has  been  little  research  published  or  conducted  into  the  ability  of  VEs  to  impart  spatial 
knowledge  of  an  area.  The  Air  Force  recently  used  VE  systems  in  Kosovo  to  conduct 
mission  rehearsal  in  a  system  strictly  designed  to  give  terrain  awareness  [THOM  99]. 
While  this  may  seem  useful  based  on  “common  sense,”  research  has  largely  not  validated 
the  concept.  If  soldiers  and  pilots  train  and  prepare  more  effectively  (as  defined  by  better 
task  performance)  with  maps  than  with  VEs,  then  it  is  a  waste  of  time  and  resources  to 
develop  such  systems  for  this  purpose.  Even  if  performance  is  better,  the  gain  may  not 
outweigh  the  increased  cost  m  time  and  money  such  a  system  inq)Oses.  Another  potential 
pitfall  is  the  lack  of  an  ideal  interface  to  train  dismounted  soldiers.  While  vehicles  confine 
soldiers  to  the  vehicle’s  limitations,  individual  soldiers  are  much  more  flexible  in  their 
actions.  VE  researchers  have  yet  to  build  an  interface  that  allows  the  soldier  to  crawl, 
walk,  and  run  through  a  conputer  environment  as  naturally  as  he  can  in  the  real  one 
[DARK  97].  This  lack  of  ideal  interface  has  not  precluded  the  inclusion  of  dismounted 
soldiers  in  models  however  [GOUR  99]. 

VEs  may  not  be  useful  in  aU  the  ways  proponents  envision.  They  may  not  prove 
useful  as  mission  rehearsal  tools,  or  as  route  rehearsal  tools.  They  may  not  assist  in 
acquiring  spatial  knowledge  of  an  area.  They  might  not  even  be  useful  as  a  tool  to  teach 
general  navigation  skills.  As  with  any  computer  training  aid,  they  may  not  actually  be  any 
better  than  the  methods  currently  in  use  [KEAR  83].  Without  knowing  the  effects  of 
computer  training,  both  positive  and  negative,  the  Army  cannot  make  accurate  decisions 
on  their  inplementation  within  the  force.  With  today’s  declining  budgets,  the  Armed 
Forces  can  ill-afford  to  waste  money  on  hunches  or  intuition. 

Before  actually  spending  money  to  contract  for  such  a  system,  it  is  critical  to 
understand  the  desired  effects  and  actual  achievable  benefits  derivable  firom  VE  training 
systems.  Without  this,  it  is  inqx)ssible  to  create  an  accurate  set  of  parameters  for  the 
models.  By  doing  this  research,  the  Army  can  determine  if  such  a  system  is  beneficial  and 
for  what  purposes  (area  familiarization,  route  rehearsal,  mission  rehearsal,  or  general  land 
navigation  training).  It  will  help  prevent  the  wasting  of  time  and  money  on  potentially 
expensive  yet  unproductive  systems  that  do  not  achieve  the  desired  benefit.  It  will  also 
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provide  a  common  set  of  standards  which  the  Army,  civilian  contractors,  and 
programmers  can  consult  during  the  development  of  such  systems. 

2.  Important  Applications  for  Spatial  Knowledge  Acquisition 

A  wide  range  of  jobs  and  activities  rely  upon  spatial  knowledge  acquisition.  Most 
people  would  readily  recognize  the  need  for  emergency  personnel  to  use  spatial 
knowledge  to  pick  the  quickest  route  to  arrive  at  the  scene  of  an  accident.  People  easily 
see  the  requirements  of  police,  firemen,  and  paramedics  to  possess  this  type  of  knowledge. 
Another  good  example  is  cab  drivers  using  their  knowledge  to  avoid  delays.  Some  of  the 
most  mundane  tasks  also  require  this  skill;  finding  specific  items  in  a  supermarket,  for 
example,  or  in  a  stadium,  locating  the  nearest  restroom  and  then  finding  your  seat  again. 

The  military,  more  than  most,  relies  heavily  on  spatial  knowledge  acquisition. 
Soldiers  and  other  military  services  use  their  knowledge  of  the  environment  to  assist  them 
in  acconplishing  their  mission,  be  that  to  attack,  retreat,  defend,  or  just  observe  [GOER 
98b].  This  requirement  is  not  limited  to  foot  soldiers.  As  seen  recently  in  the  Kosovo  air 
war,  pilots  too  need  accurate  knowledge  of  their  operating  environment  to  successfully 
acconplish  their  task  in  a  safe  and  efficient  manner. 

In  short,  any  activity  that  requires  movement  through  space  or  through  conplex 
environments  does  in  fact  require  spatial  knowledge  acquisition.  These  can  be  military  or 
civilian  but  they  are  pervasive  throughout  our  lives. 

3.  Spatial  Knowledge  Acquisition  as  a  Proven  Concept 

Researchers  have  extensively  studied  spatial  knowledge  acquisition  because  it  is 
such  a  prevalent  and  important  requirement  throughout  the  scope  of  human  activity.  The 
most  accepted  model  of  human  ability  to  acquire  detailed  spatial  knowledge  is  provided  by 
Thomdyke  who  created  a  sinplified  model  to  explain  the  process  [THOR  80]  (see  Chapter 
n  Spatial  Knowledge  Acquisition).  Since  we  know  that  humans  can  acquire  spatial 
knowledge  about  real  world  areas,  researchers  next  studied  if  humans  could  do  the  same 
with  VEs  [GILL  97]  [RUDD  98].  Following  this,  they  also  examined  if  this  spatial 
knowledge  could  be  transferred  fi'om  the  VE  to  the  real  world  area  [WITM  95]  [BLIS  97] 
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[DARK  98]  [WALL  98].  Because  this  concept  was  proven  for  VEs  in  general  and  the 
transfer  from  VE  to  the  real  world  in  man-made  indoor  environments,  it  was  natural  to 
next  explore  if  this  previous  research  holds  valid  when  moved  to  natural  unstructured 
outdoor  environments. 

4.  Existing  Research  Shortcomings 

While  Chapter  n  discusses  much  of  the  existing  background  research  for  the 
thesis,  several  points  are  highlighted  here.  Many  problems  exist  with  current  CBTs 
because  there  is  a  lack  of  research  to  back  up  their  purpose.  There  is  little  proof  that  using 
certain  CBTs  is  any  better,  if  not  worse,  than  current  techniques.  Furthermore,  where 
there  is  proof,  it  is  often  the  cost  savings,  not  the  benefit  gain  that  is  the  advantage.  For 
exanple,  only  about  59%  of  tasks  trained  in  a  flight  simulator  carry  over  into  the  real 
world,  but  at  10%  of  the  cost  of  operating  the  actual  plane  [ANGI 93].  Many  groups  seem 
eager  to  junqp  onto  the  technology  bandwagon  and  do  not  first  see  whether  using  the  CBT 
results  in  “better”  task  performance.  Users  should  justify  any  shift  towards  CBT  in  terms 
of  cost  or  performance  gain. 

As  conoputer-processing  power  has  improved,  so  too  has  the  achievable  level  of 
fidelity  in  VEs.  They  can  be  more  accurate  and  more  realistic  looking  than  in  past  years. 
There  has  been  much  effort  spent  on  increasing  fidelity  in  the  models,  but  little  research  to 
show  how  this  increased  fidelity  affects  performance.  Determining  the  optimal  level  of 
fidelity  will  be  critical  for  building  future  VEs  that  are  efficient  for  training  skills.  More 
research  into  the  area  of  fidelity  is  needed. 

Because  the  study  of  transfer  of  spatial  knowledge  from  VEs  to  the  real  world  is 
relatively  new,  there  are  many  questions  remaining  unanswered.  The  most  inportant  is 
how  to  maximize  the  transfer.  In  an  ideal  case,  if  a  person  performs  the  task  in  a  VE,  it 
would  have  the  same  learning  affect  as  if  he  did  it  in  the  real  world.  Questions  as  to  the 
ideal  interface,  display,  fidelity,  and  movement  within  the  virtual  environment  remain 
unanswered.  VEs  provide  the  user  with  the  ability  to  do  things  in:5)ossible  in  the  real  world 
(e.g.  travel  at  any  speed,  fly,  and  teleportation)  but  the  most  effective  combination  of  these 
tools  is  unknown.  Do  these  tools  aid  learning,  or  are  they  distractions?  Finally,  there  is  a 
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lack  of  research  extending  the  transfer  of  spatial  knowledge  from  VEs  to  the  real  world  in 
natural  environments.  This  thesis  intends  to  address  that  issue. 


C.  THESIS  ORGANIZATION 

The  thesis  is  organized  as  follows:  Chapter  n  provides  background  of  the 
mihtary’s  use  of  simulations,  spatial  knowledge  acquisition,  learning  techniques,  previous 
studies  on  the  use  of  VEs,  and  the  work  of  prior  students  that  this  thesis  builds  upon. 
Chapter  III  discusses  the  VE  that  MAJ  Simon  Goerger  developed  and  used  in  this  thesis’ 
experiments.  It  also  presents  the  details  of  all  the  tests,  took,  and  methods  used  in  the  two 
experiments.  This  section  ako  discusses  the  two  different  experimenits  conducted  as  part 
of  the  thesk  and  their  respective  methodologies.  Chapter  IV  analyzes  the  data  collected 
from  the  two  experiments  and  dkcusses  the  results.  It  ako  compares  the  results  to  the 
original  Goerger  experiment.  Chapter  V  con^ares  the  findings  of  the  two  experiments  and 
the  original  Goerger  study,  dkcusses  their  impact,  and  lays  out  future  areas  of  interest  for 
follow-on  research. 

The  thesk  includes  numerous  appendices  that  show  the  maps,  partic^ant 
instructions,  and  experiment’s  timelines.  Information  concerning  route  difficulties  and 
common  land  navigational  terms  are  ako  included  as  appendices.  In  addition  the  raw  data 
from  the  experiments  k  included. 
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n.  BACKGROUND  AND  PREVIOUS  WORK 

A.  THE  MILITARY  AND  MODELS/SIMULATIONS 

1.  Historical  use  of  Simulations  for  Training 

The  military  use  of  wargames  or  other  simulations  for  training  is  only  slightly 
younger  than  the  history  of  organized  warfare  itself.  Archeologists  have  discovered  groups 
of  miniature  Sumerian  soldiers  in  formation,  as  well  as  later  Egyptian  units  [PATR  77]. 
While  these  may  indeed  have  started  as  mere  toys  for  the  nobility,  at  some  point  the  play 
with  these  soldiers  became  stylized,  evolving  into  games  such  as  Go  and  Chess.  While  the 
value  of  Chess  as  a  direct  simulation  has  obviously  faded  far  in  the  past,  the  spirit  lives  on. 
In  1780,  Helwig  created  the  first  wargame  to  expand  beyond  the  set  “chess-style” 
boundaries.  It  included  1666  squares  representing  different  terrain,  and  players  had  120 
units  per  side,  including  cavalry,  forts,  and  artillery. 

In  1811,  Herr  Von  Reisswitz  did  away  with  the  idea  of  a  game  board  entirely, 
introducing  a  game  played  on  a  sand  table,  where  terrain  matched  a  noap  at  1:2373  scale. 
His  son  later  transformed  the  game  to  maps  at  1:8,000  scale  and  in  1824  introduced  it  to 
then  Chief  of  Staff  of  the  Prussian  Army  General  Von  Muffling,  calling  it  Kriegspiel  (war 
play).  After  the  demonstration.  Von  Muffling,  who  had  been  unenthusiastic  before, 
exclaimed  “It  is  not  a  game  at  all,  it’s  training  for  war!  I  shall  recommend  it  most 
emphatically  to  the  whole  army  [PATR  77].”  One  of  the  major  evolvements  of  the 
Reisswitz  wargame  was  the  use  of  un^ires.  These  neutral  parties  were  used  to  provide 
fog-of-war  and  resolve  disputes.  Wargames  became  even  more  popular  in  Prussia  under 
Von  Molke.  After  the  stunning  German  victories  in  1870,  many  nations  rushed  to  include 
wargaming  into  their  training  programs  as  well.  The  Japanese  in  particular  credit  their 
victory  in  the  Russo-Japanese  war  of  1904  m  part  to  their  wargaming. 

Within  the  United  States,  the  military  first  picked  up  on  wargaming  in  1867  and  by 
1882  the  American  Army  produced  it’s  own  wargame.  The  American  Kriegspiel,  A  Game 
for  Practicing  the  Art  of  War  Upon  a  Topographical  Map.  Later  the  American  Army 
moved  to  force-on-force  engagements  that  were  more  involved  than  the  staff  room 
training  STX/CTXs  (situational  training  exercises/combat  training  exercises).  Here  units 
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maneuvered,  dug  in,  and  actually  did  everything  but  shoot.  Historical  events  such  as  the 
Louisiana  Maneuvers  prior  to  World  War  n  and  the  military’s  annual  REFORGER 
exercises  in  the  1970/1980’s  are  of  this  type.  The  US  Army  even  developed  a  Tnannal  for 
its  umpires,  FM105-5  Maneuver  Control,  to  standardize  rules  about  casualties,  advance 
rates,  etc.  One  problem  with  this  system  is  that  often  the  training  would  break  down  into 
“I  shot  you,  no  you  didn’t”  disagreements  more  reminiscent  of  children  playing  than  an 
army  training. 

With  the  progress  of  technology,  especially  the  introduction  of  the  MILES 
(Multiple  Integrated  Laser  Engagement  System)  system,  the  Army  was  able  to  remove  the 
umpire  from  direct  involvement  in  actual  engagement  training.  This  made  the  training 
more  realistic  and  solved  the  “I  shot  you”  problem.  Now,  the  sensors  determine  if  a  hit 
was  scored  or  not.  It  allowed  for  better  training  at  night  where  before  the  umpires  had 
been  hampered.  Different  laser  signatures  were  used  for  different  weapons  so  that  a  rifle 
could  not  destroy  a  tank.  The  rise  of  SIMNET  (see  Section  A.3)  allowed  the  fighting  to  be 
removed  from  the  field  and  into  computers. 

Currently,  the  US  Army  defines  three  different  types  of  models  and  simulations 
that  support  training;  live,  constructive,  and  virtual.  Live  simulations  are  those  that 
involve  personnel  and  vehicles  in  the  field.  These  may  involve  umpires  or  the  MILES 
system  to  assist  m  results  [DMSO  93].  Constructive  simulations  hearken  back  to  the 
original  wargames,  and  are  used  in  staff  planning  and  training  exercises.  They  are  useful  in 
simulating  large  conflicts  across  a  region,  often  involving  the  entire  theater  [DMSO  93]. 
Currently  the  Army  uses  such  systems  for  its  BCTP  (Battle  Command  Training  Program) 
and  its  WARFIGHTER  exercises  (evaluation  exercises  for  Divisional  Staffs).  The  final 
classification  of  simulation  is  Virtual.  The  SIMNET  and  Close  Combat  Tactical  Trainer 
(CCTT)  programs  fall  into  this  category  (see  Section  A.3). 

Why  do  units  conduct  simulations?  General  Gorman,  the  former  commander  8* 
Infantry  Division,  said  ‘The  first  battle  of  most  wars  fought  by  the  Army  of  the  United 
States  was  a  disaster:  a  costly  defeat  or  a  Pyrrhic  victory.”  [GORM  90]  He  added  that 
simulations  that  can  be  rehearsed  repeatedly  to  increase  combat  readiness,  refine  combat 
skills,  and  protect  the  force  before  actual  combat  are  invaluable.  The  goal  is  to  mpart 
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combat  experience  without  the  actual  dangers  of  combat.  In  studies  of  real  combat  results, 
inexperienced  soldiers  and  pilots  suffered  higher  causality  rates,  but  as  they  became  more 
experienced,  their  survivability  in^roved.  The  use  of  simulations  for  training  can  have 
dramatic  mq)act  upon  battlefield  results.  For  exan5)le,  during  the  Vietnam  War,  one  US 
fighter  was  lost  for  every  two  North  Vietnamese  fighters.  This  ratio  inq)roved  to  12.5  to  1 
after  the  Navy  implemented  its  TOP  GUN  program,  which  was  an  engagement  simulation 
[GORM  90].  The  best  reason  behind  using  simulations  is  that  ‘The  bottom  line  is  it 
prepares  us  to  be  the  best  we  can.”  [BELC  99] 

2.  Model  Usage  for  Mission  Planing  and  Preparation 

Throughout  military  history,  forces  have  incorporated  detailed  rehearsals  into  their 
preparation  for  coming  battles.  The  US  Army’s  Ranger  Handbook,  the  bible  of  its  infantry 
forces,  states  ‘They  [rehearsals]  are  essential  to  ensure  con:5)lete  coordination  and 
subordinate  understanding.”  The  handbook  adds  that  they  should  be  conducted  in  an  area 
as  similar  to  the  objective  as  possible.  The  rehearsals  should  include  brief-backs  over  sand 
tables,  or  sketches  [RHB  95].  While  many  of  these  rehearsals  may  involve  merely  smq)le 
checklists,  or  talk-through  of  the  planned  battle,  others  are  more  detailed.  These  may 
involve  the  use  of  sand  tables,  map-boards,  dioramas,  simple  dirt  maps,  or  even  elaborate 
mock-ups,  to  better  picture  the  terrain,  as  well  as  the  individual  unit’s  locations  in  time 
and  space.  For  many  operations,  the  military  would  actually  build  a  life  size  model  of  the 
target  so  units  could  practice  moving  about  the  ground.  This  was  what  the  Israeli  Army 
did  in  its  preparation  for  the  Entebbe  airport  raid.  They  laid  out  the  terminal  using  metal 
poles  and  burlap  for  walls  to  exact  dimensions  [MCRA  95].  The  troops  were  also  shown 
video  footage  of  the  terminal  area  (taken  fi^om  a  soldier’s  personal  vacation  footage). 

f 

Sixnilar  missions  did  not  always  have  similar  preparation  time  but  always  included 
at  least  some  form  of  rehearsal.  Consider  the  US  Ranger  raid  on  Cabanatuan  in  January 
1945  in  the  Philippines  in  WWQ  and  the  Son  Tay  raid  during  the  Vietnam  War.  The 
rangers  liberating  the  Philippines  had  crude  dirt-maps  and  sand  table  briefings  conducted 
at  patrol  bases  within  gunshot  of  the  enemy.  They  had  less  than  three  days  to  prepare  their 
mission  and  much  of  that  time  was  spent  near  the  prison  canq)  some  70  miles  behind 
enemy  lines.  Despite  this,  their  final  preparations  involved  briefing  aU  soldiers  over  the 
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canp  layout  and  objectives  based  on  the  most  recently  scouted  information.  The  forces 
raiding  Son  Tay  trained  extensively  for  months  on  elaborate  mock-ups  [MCRA  95].  The 
entire  complex  was  replicated  in  exacting  detail  for  rehearsals,  as  well  as  all  other  aspects. 

During  World  War  H,  the  Army  Air  Corps  prepared  its  pilots  for  air  raid  missions 
over  Japan  by  showing  them  films  of  the  precise  routes  they  would  be  flying  over  enemy 
territory.  These  fiilms  were  not  produced  firom  satellite  imagery  or  over  flights  by 
American  reconnaissance  aircraft.  Instead,  the  films  were  produced  by  the  Air  Corps  film 
and  production  unit  stationed  in  Hollywood,  California  [AMER  98].  The  production  unit 
built  a  model  of  Japan  using  over  fifty  ten  foot  square  platforms,  tons  of  plywood, 
modeling  clay,  burlap,  and  paint.  Using  reconnaissance  photos  of  the  island,  crews  worked 
twenty-four  hours  a  day  for  weeks,  expending  thousands  of  man-hours,  to  construct  and 
paint  an  exacting  replica  of  the  island  so  that  camera  crews  could  film  bombing  routes  for 
pilots.  In  order  to  maintain  security,  the  model  was  built  and  filmed  entirely  on  a  single 
sound  stage  that  was  placed  off  limits  to  everyone  except  the  personnel  working  on  the 
project.  Pilots  routinely  conunented  on  how  easy  it  was  to  recognize  the  terrain  as  they 
flew  their  noissions  because  it  was  as  if  they  had  been  there  before. 

Many  of  the  models  used  in  recent  military  operations  came  from  the  CIA’s 
Modeling  Shop  [FINN97].  This  group  specializes  in  creating  detailed  replicas  of  various 
sites  and  structures.  They  helped  in  the  Son  Tay  raid  in  1970,  and  made  replicas  of  the  US 
Embassy  in  Iran  and  the  Kremlin  in  Moscow.  The  hours  and  expense  in  making  these 
models  was  immense  but  considered  valuable  based  on  the  requirement  for  detailed 
exposure  to  the  environment  prior  to  the  mission.  However,  these  models,  in  all  their 
detail,  lacked  the  ability  to  actually  walk  into  the  stracture  and  move  around.  The 
Modeling  Shop  shifted  to  making  con^uter  generated  models  in  1997  [FINN  97]. 

3.  The  Military  and  VEs 

The  current  military  is  smaller  than  at  any  time  since  WWII  and  has  shifted  from  a 
forward-deployed  force  to  a  power  projection  one  [DMSO  93].  The  reduction  in  size  and 
shift  m  missions  has  forced  the  military  to  change  its  training  and  doctrine.  The  military 
has  become  very  interested  in  using  simulations  and  VE  in  particular  to  augment 
traditional  training. 
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As  stated  above,  the  military  had  always  found  simulations  useful,  but  the  advent 
of  VEs  allowed  them  to  take  advantage  of  the  new  technology.  Among  the  first  systems 
built  were  single  vehicle  trainers.  Flight  simulators  were  developed  to  allow  pilots  to  train 
at  a  firaction  of  the  cost  of  actually  flying  jets.  The  Army  developed  the  UCOFT  system 
(Unit  Conduct  of  Fire  Trainer),  which  was  designed  to  train  tank  crews  in  gunnery 
operations.  The  fact  that  the  UCOFT  provided  improvements  in  gunner  proficiency  was 
seen  as  validation  of  the  use  of  simulators  for  non-aerial  vehicles  [BOLD  85]. 

The  Army  places  great  en^hasis  on  training  as  combined  arms  or  “training  as  we 
fight”[FM25  88].  While  single  vehicle  simulators  worked,  the  military  fights  as  units  and 
teams,  so  the  military  was  looking  for  systems  that  would  allow  these  groups  to  be 
brought  together  for  collective  training.  Networking  simulators  were  the  result  of  Defense 
Advanced  Research  Project’s  Agency  (DARPA)  sponsored  technology  demonstration 
projects  [DMSO  93]  and  were  the  obvious  answer  to  the  collective  training  problem. 
Among  the  products  were  those  for  the  Army’s  tank  and  infantry  fighting  vehicle 
(SIMNET).  Later  systems  were  added  for  A-lOs  and  attack  helicopters.  These  systems 
brought  the  individual  vehicle  simulators  into  one  virtual  world  and  allowed  them  to 
maneuver  together  in  real-time.  This  allowed  the  units  to  practice  together  in  the 
simulators  making  them  much  more  useful. 

The  Army’s  latest  development  is  the  Close  Combat  Tactical  Trainer  (CCTT). 
Using  high  fidelity  maimed  modules,  it  represents  combat  between  armored  vehicles  and 
dismounted  infantry.  The  system  is  not  designed  to  be  an  individual  skills  trainer  [DMSO 
93],  rather  one  that  trains  teams,  platoons,  and  con^ianies.  The  goal  of  the  system  is  to 
allow  the  platoon  through  task  force/battalion  level  to  train  on  collective  tasks  cheaper  and 
more  safely  than  they  can  in  the  field  [GOUR  99].  While  not  designed  to  replace  field 
training,  it  will  be  used  to  augment  it,  as  training  dollars  become  short.  The  envisioned 
advantage  over  the  current  SIMNET  system  is  a  much  higher  fidelity  level,  allowing  for 
the  inclusion  of  realistic  smoke,  noise,  and  debris  'effects  [DMSO  93].  This  will  make  the 
training  more  realistic  and  follow  the  guidance  of  FM25-100  Training  the  Force,  which 
says  all  training  should  strive  to  be  as  realistic  as  possible  [FM25  88].  The  goal  is  that  use 
of  this  system,  while  saving  money,  will  not  reduce  the  realistic  training  conditions.  Note 
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that  the  DOD  has  no  evidence  that  the  inclusion  of  these  effects  will  result  in  better 
transfer  of  task  learning,  nor  in  increased  performance.  This  follow-on  to  SIMNET  is 
hoped  to  be  fully  functional  by  2003. 

Another  system  of  note  currently  in  use  is  the  Topscene  system.  “  Topscene  is  an 
aircraft  simulation  using  real-time  and  real  imagery  to  conduct  mission  planning,  mission 
rehearsal,  and  pilot  training”[DMSO  93].  The  system’s  primary  use  has  been  in  visualizing 
threat  radiuses,  planning  egress  routes,  and  terrain  masking.  It  is  not  a  combat  simulation 
but  more  of  a  planning  simulation.  Recently,  pilots  in  Kosovo  used  TOPSCENE  during 
the  conflict  prior  to  conducting  their  missions  [THOM  99].  While  there  is  no  statistical 
data  to  back  their  opinions  up,  pilots  say  it  helps  them  identify  landmarks  and  assists  in 
mission  rehearsal  [THOM  99]. 


B.  SPATIAL  KNOWLEDGE  ACQUISITION 

Spatial  knowledge  or  spatial  cognition  is  a  mental  representation  of  a  real  or 
virtual  environment  [WICK  92].  Thomdyke  theorized  that  there  were  three  levels  of 
spatial  knowledge:  Landmark,  Route,  and  Survey  Knowledge  [THOR  80]. 

Landmark  knowledge  is  defined  as  being  able  to  recognize  distinctive  features  or 
locations  located  at  a  specific  location  in  an  area.  Landmark  knowledge  is  the  ability  to 
memorize  features  in  an  environment,  such  as  specific  hills,  road  intersections,  or 
buildings.  Later,  this  knowledge  is  able  to  be  recalled  allowing  individuals  to  quickly 
recognize  the  feature  or  location  when  they  see  it.  Landmark  knowledge  is  most  often 
acquired  by  direct  exposure  to  an  environment,  though  it  is  also  possible  to  gain  it  through 
study  of  a  map  or  photograph  of  the  area.  Individuals  successfully  demonstrate  landmark 
knowledge  by  their  ability  to  recognize  distinct  locations  or  unique  objects  within  an  area 
[DARK  95]  [THOR  80].  Figure  2.1  shows  an  example  of  an  individual  with  landmarks 
knowledge  of  an  area.  The  subject  would  recognize  that  the  area  has  three  distinct  features 
X,  Y,  and  Z  and  will  recognize  these  features  if  he  sees  them.  The  subject  would  not 
necessarily  know  any  information  concerning  the  relative  positions  of  these  features. 
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Figure  2.1.  Landmark  Knowledge 


Route  knowledge  is  identified  as  the  ability  to  navigate  along  a  route  or  path 
between  landmarks  or  distant  locations  [GOIX  91].  It  expands  the  recognizing  of  specific 
areas  from  landmark  knowledge  into  a  more  complex  arrangement  of  linking  those  areas 
by  a  path  or  route.  Route  knowledge  is  derived  from  an  egocentric  (inside-out)  viewpoint 
and  is  characterized  by  being  able  to  move  from  one  landmark  to  the  next  following  a 
prescribed  path.  Like  landmark  knowledge,  route  knowledge  is  gained  through  repeated 
exposure  to  an  environment  or  through  the  study  of  a  map  or  overhead  photograph.  The 
exposure  to  a  video  route  of  the  environment  has  also  been  shown  to  develop  this 
knowledge  [GOLD  82].  This  was  also  shown  during  WWII  by  the  Army  Air  Corps’ 
produced  “films”  of  bomber  routes  over  Japan.  The  pilots  credited  these  films,  which 
showed  the  pilots  the  routes  they  would  follow  on  their  attack,  with  making  it  easy  to 
confirm  they  were  in  the  right  place.  The  fact  that  these  films  were  not  of  the  actual 
mainland  but  were  created  from  an  extensive  model  had  no  effect  on  the  results 
[AMER98].  Figure  2.2  shows  an  exanqrle  of  an  individual  with  route  knowledge  of  an 
area.  The  subject  not  only  has  knowledge  of  certain  landmarks  and  distinctive  features  (X, 
Y,  and  Z),  but  they  also  have  the  knowledge  to  traverse  from  one  landmark  to  another 
along  at  least  one  set  path  (the  route  from  X  to  Y  and  then  to  Z). 


Figure  2.2.  Route  Knowledge 


Route  Knowledge  does  not  necessarily  confer  the  ability  to  conduct  the  reverse  of 
the  learned  route,  nor  does  it  mean  they  may  know  of  alternate  routes  or  shortcuts.  It  does 
not  imply  that  the  subject  knows  the  orientation  of  Y  from  X,  just  that  they  know  a  route 
or  path  on  how  to  travel  from  one  to  the  other  [KOH  97]. 

The  final  and  highest  level  of  spatial  knowledge  is  survey  (or  configurational) 
knowledge.  It  represents  a  naap-like  or  top  down  mental  encoding  of  the  environment  and 
is  based  on  an  exocentric  (outside-in)  viewpoint.  In  survey  knowledge  an  individual  can 
not  only  recognize  specific  locations  or  landmarks,  but  can  accurately  place  them  in  his 
environment  even  if  he  cannot  see  them  Individuals  can  also  traverse  throughout  their 
area  without  having  to  pre-plan  the  exact  route  because  they  know  the  layout  of  the 
region  [BANK  97].  Because  of  its  exocentric  nature,  survey  knowledge  is  most  often 
gained  through  map  or  aerial  photograph  study.  Extensive  exposure  to  an  area  has  also 
been  demonstrated  to  develop  this  knowledge  [THOR  80].  Figure  2.3  shows  an  exaiqple 
of  an  individual  with  survey  knowledge  of  an  area.  The  subject  can  describe  the  relative 
distances  and  locations  of  all  major  objects  in  the  area  in  relation  to  the  subject  and  each 
other.  The  subject  at  d  can  tell  how  far  he  is  from  X,  Y  and  Z,  even  if  he  cannot  see  them 
He  can  also  plot  a  route  to  any  feature  without  needing  prior  study. 


Figure  2.3.  Survey  Knowledge 


Thomdyke’s  theory  on  the  process  by  which  humans  acquire  spatial  knowledge, 
displayed  in  Figure  2.4,  is  generally  accepted.  It  explains  how  humans  use  the  spatial 
information  of  their  surroundings  to  create  a  mental  map  or  representation  of  the  world 
around  them  Thomdyke  explains  how  each  level  is  not  self-inclusive,  but  in  fact  builds 
upon  the  knowledge  of  previous  levels  [THOR  80]. 
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Figure  2.4.  Navigation  Knowledge 


C.  VIRTUAL  ENVIRONMENTS 

1.  Definitions  of  VE 

In  order  to  discuss  the  effectiveness  of  VEs  it  is  important  to  have  an 
understanding  of  several  of  the  major  component  definitions. 

Any  model  which  provides  firame  rates  of  eight  to  ten  firames  per  second  (^s)  for 
static  environments  and  60  ^s  for  more  dynamic  environments  can  be  said  to  have  real¬ 
time  graphics  [DURL  95].  Real-time  graphics  over  a  network  also  must  have  a  minimum 
of  network  latency  (less  than  0.1  seconds). 

Fidelity  is  more  a  qualitative  than  a  quantitative  classification.  No  defined  scales  or 
agreed  upon  distinctions  exist.  Waller  et  al.  described  environmental  fidelity  as  the  degree 
to  which  the  variables  in  the  model  matched  those  in  the  real  world  [WALL  98].  Goerger 
defined  high  fidelity  as  “a  model  that  represents  lines  of  sight  and  terrain  masking, 
provides  realistic  depictions  of  the  vegetation  and  structures,  and  can  provide  a  real-time 
interactive  environment  for  the  user”  [GOER  98b].  Past  studies  have  shown  that 
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increasing  the  fidelity  of  a  model  leads  to  an  inq)roved  transfer  to  the  real  world  [HAYS 
89][CAIR  96],  but  these  results  do  not  concern  outdoor  environments  in  particular.  For 
proving  the  validity  of  the  transfer  of  spatial  knowledge,  Goerger  wanted  to  use  as 
realistic  looking  a  model  as  possible  [GOER  98b].  Exanq)les  of  the  model  used  in  this, 
research  and  the  corresponding  real  world  photo  are  shown  in  figures  2.5  and  2.6. 


Figure  2.5.  Model  Photo  Figure  2.6.  Real  World  Photo 


Goerger  described  landmark  models  as  “virtual  representations  of  real  world 
objects  or  locations  that  are  easily  identified,  with  defining  characteristics,  and  are  used  by 
the  participants  as  cues  to  navigate  through  the  model”  [GOER  98b].  These  are  in  effect 
key  landmarks,  which  must  be  included  m  any  model  built  of  a  particular  environment  if 
that  model  is  to  allow  the  acquisition  of  landmark  knowledge  based  on  Thomdyke’s 
theory.  Failure  to  include  these  essential  features  could  result  in  the  model’s 
ineffectiveness  in  representing  an  area  and  potentially  inhibit  the  acquisition  of  landmark 
knowledge,  which  in  turn  may  inhibit  route  and  survey  knowledge  development. 

2.  Prior  Studies  of  Spatial  Knowledge  and  Virtual  Environments 

Chase  examined  the  differences  between  individuals  who  had  significant  exposure 
to  an  environment,  but  had  never  seen  a  map  or  exocentric  view  of  it,  versus  those  who 
had  only  seen  a  map  but  had  no  direct  exposure  to  the  environment  [CHAS  83].  In  the 
study,  those  with  no  direct  exposure  tended  to  have  better  survey  knowledge,  while  those 
with  only  direct  exposure  tended  to  display  better  landmark  and  route  knowledge.  rha«t 
reported  that  the  repeated  exposure  to  an  area  develops  landmark  and  route  knowledge, 
but  that  this  exposure  does  not  automatically  develop  survey  knowledge.  The  results 
showed  that  use  of  a  map  provided  the  best  survey  knowledge  but  that  the  repeated 
exposure  group  outperformed  the  map  group  in  landmark  and  route  knowledge 
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development.  Chase’s  findings  support  how  Thomdyke  believed  spatial  knowledge  to  be 
acquired  (see  Section  B.  Spatial  BCnowledge  Acquisition).  The  use  of  a  map  to  rapidly 
acquire  survey  knowledge  of  an  environment  without  prior  exposure  to  it  is  also  reported 
by  Hirtle  and  Hudson  [HIRT91]. 

Bliss,  et  al.  examined  the  use  of  VEs  in  acquiring  spatial  knowledge  [BLIS  97]. 
They  wanted  to  see  if  exposure  to  a  VE  was  the  same  as  exposure  to  the  real  world.  They 
broke  up  30  firefighter  subjects  into  three  groups  and  experimented  performing  simulated 
rescue  operations  in  an  office  building.  One  group  was  map-only  and  was  given  a  map  to 
study  filrst.  A  second  group  could  first  explore  the  building  with  a  VE.  The  final  group  (the 
control)  was  given  no  training  at  all.  Both  the  Map  and  VE  groups  clearly  performed 
better  than  the  control  group  in  the  results.  Though  the  experimenters  had  tests  and 
performance  measure  for  landmark  and  route  knowledge,  they  had  no  corresponding  test 
for  survey  knowledge.  Nevertheless,  they  concluded  from  their  results  that  landmark, 
route,  and  survey  knowledge  could  be  gained  from  VEs.  Their  research  did  not  show  any 
advantage  for  the  VE  over  the  map. 

Waller,  Hunt,  and  Knapp  conducted  a  series  of  studies  on  the  transfer  of  spatial 
knowledge  in  virtual  environment  testing  [WALL98].  They  devised  a  test  involving  the 
exploration  of  a  real  world  naaze.  Subjects  were  divided  into  six  different  groups.  The 
blind  group  had  no  exposure  prior  to  the  test.  The  real  world  group  had  one  minute  to 
explore  while  the  map-only  group  had  one  minute  to  study  a  map.  Three  different  VE 
groups  were  used:  a  desktop  based  VE,  an  immersive  VE  using  a  head  mounted  display, 
and  a  long-term  immersive  VE.  Both  the  desktop  and  immersive  VE  groups  were  given 
two  minutes  to  study.  The  long-immersive  group  was  given  five.  The  results  indicated  that 
the  low  fidelity  VE  system  used  did  allow  subjects  to  “develop  useful  representations  of  a 
large  scale  navigable  space”[WALL  98].  The  study  found  that  with  only  short  exposure, 
VE  training  was  not  more  effective  than  map  training.  However,  after  significant 
exposure,  the  long-term  VE  group  did  outperform  the  map  group. 

On  the  subject  of  the  effect  of  immersion  and  presence  on  training,  the  results  are 
noixed.  Ruddle,  Randal,  Payne,  and  Jones  showed  no  significant  differences  between  the 
performance  of  immersive  and  non-immersive  VEs  groups  in  exploring  virtual  buildings 
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[RUDD  96].  Wickens  and  Prevett  showed  that  aviators  who  had  an  immersive  viewpoint 
had  better  navigational  performance  over  those  who  were  given  external  non-immersive 
views  [WICK  95].  This  is  possibly  due  to  the  more  natural  representation  or  intuitive 
viewer  interpretation  based  on  the  field  of  view  [WICK  98].  Other  studies  have  shown 
that  users  estimate  distance  much  more  accurately  if  they  are  in  an  immersive  VE  where 
they  physically  rotate  than  if  in  one  where  they  merely  imagine  they  rotate  [PRES  94]. 

Williams,  et  al.  investigated  the  area  of  active  versus  passive  control  during  flight 
mission  preparation  and  its  effect  on  performance  [WILL  95].  The  experiment  looked  at 
those  subjects  who  actively  controlled  their  flight  of  the  aircraft  in  the  VE  versus  those 
who  passively  watched  a  flight.  The  results  indicated  that  those  subjects  actively  involved 
performed  better  than  those  who  merely  observed.  The  researchers  concluded  fi’om  their 
data  that  the  optimal  VE  designed  to  acquire  spatial  awareness  would  include  active 
control  by  the  user  and  not  merely  passive  playback. 

Witmer’s  study  into  the  transfer  of  spatial  knowledge  turned  up  useful  information 
regarding  the  inportance  of  the  user  interface  [WITM  95].  He  examined  64  subjects  in 
navigating  a  large  building.  He  divided  his  subjects  into  three  groups:  a  real  world  group,  a 
VE  group,  and  a  verbal  directions  only  group.  His  results  showed  the  VE  group 
performing  poorer  than  the  real  world  group,  but  ahead  of  the  verbal  directions  group. 
The  results  also  indicated  that  spatial  skills  learned  in  a  VE  could  transfer  to  the  real  world 
in  some  cases.  Some  VE  subjects  had  their  training  seriously  inpeded  by  difficulties  with 
the  interface.  These  same  individuals  had  difficulty  performing  the  navigation  task  in  the 
actual  building,  lending  to  speculation  that  poor  interfaces  can  diminish  the  training 
effectiveness  of  a  VE  [WITR  95]. 

Darken  and  Sibert  have  conducted  several  experiments  on  wayfinding  and  virtual 
environments.  In  “Navigating  Large  Virtual  Spaces”  [DARK  96b]  they  concluded  that  in 
exploring  a  large  VE  without  aids,  subjects  quickly  became  lost  and  disoriented.  They 
found  that  inclusion  in  the  model  of  a  map,  con^ass,  or  grid  helped  overcome  this 
difficulty.  They  also  found  that  the  map  provided  superior  navigational  performance  while 
the  grid  provided  superior  directional  performance  [DARK  96b].  In  “Wayfinding 
Strategies  and  Behaviors  in  Large  Virtual  Worlds”  [DARK  96a]  they  determined  that 
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individuals  provided  with  supplementary  aids  in  the  VE,  such  as  maps,  were  able  to  more 
quickly  regain  their  starting  position  and  maintain  their  orientation  than  when  no  aids  were 
provided.  They  also  noted  that  when  exploring  a  VE,  subjects  tended  not  to  revisit  places 
they  had  already  been. 

In  the  first  iteration  of  this  thesis  experiment.  Banker  theorized  that  a  combination 
of  map  study  and  VE  exposure  would  provide  the  optimal  solution  for  providing  total 
spatial  knowledge  of  an  environment.  Using  a  non-real-time  model,  he  concluded  that 
subjects  of  intermediate  ability  could  successfully  gain  and  transfer  spatial  knowledge  fi’om 
a  VE  to  a  real-world  outdoor  environment  [BANK  97].  Banker’s  study  had  three  different 
groups:  map-only,  real  world  and  VE.  All  of  the  subjects  were  active  in  orienteering 
though  of  different  ability  levels.  None  of  the  subjects  was  familiar  with  the  specific  testing 
area  though  many  had  been  in  similar  terrain.  Subjects  were  given  one  hour  of  study  with 
which  to  plan  and  memorize  a  route  through  the  environment.  All  of  the  subjects  had 
exposure  to  the  map  during  their  study  phase.  After  study,  subjects  were  taken  to  the 
coiurse  and  had  to  run  their  route  from  memory.  Banker's  results  showed  that  ability  level 
had  a  greater  iiiq)act  on  the  results  than  method  of  training,  with  advanced  level  subjects 
doing  best  overall.  He  noted  a  significant  increase  in  performance  for  those  classified  as 
intermediate  level  orienteerers.  Little  increase  in  performance  was  noted  in  the  advanced 
level  or  beginning  level.  Banker  theorized  that  the  ability  and  experience  of  advanced 
orienteerers  allowed  them  to  gam  significant  information  fi'om  the  map,  which  was  not 
greatly  augmented  by  the  VE.  He  also  concluded  that  the  beginners  were  overwhelmed 
with  information  and  too  inexperienced  to  correctly  focus  on  which  information  to  study 
for  success. 

Goerger,  et  al.  conducted  two  studies  similar  to  the  Banker  experiment.  The  filrst 
experiment  used  a  complex  man  made  environment  to  con:q)are  two  groups:  a  map-only 
study  group  and  VE  study  group.  The  study  followed  a  similar  pattern  to  the  Banker 
experiment  [GOER  98a].  Thirty  minutes  were  given  to  both  groups  to  study  floor  plans  of 
the  seven-story  structure.  The  VE  group  also  had  a  high  fidelity  real-time  computer 
representation  of  the  building,  which  they  could  explore  during  the  thirty-minute  study 
phase.  The  results  of  the  experiment  showed  the  map-only  group  significantly 
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outperformed  the  VE  group.  Goerger  theorized  that  the  short  exposure  limited  the 
performance  of  the  VE  group.  He  concluded  that  “performance  on  spatial  knowledge 
tasks  after  brief  exposure  to  a  high  fidelity,  real-time  VE  does  not  always  exceed  results 
gained  from  traditional  navigation  training  techniques”  [GOER  98a].  This  theory  is  backed 
up  by  the  results  of  Waller,  Hunt,  and  Knapp.  They  found  “short  periods  of  VE  training 
were  no  more  effective  than  map  training;  however,  with  sufficient  exposure  to  the  virtual 
training  environment,  VE  training  eventually  supersedes  the  map”  [WALL  98].  Ruddle,  et 
al.  also  found  users  performed  better  the  longer  they  used  the  system  and  the  more  familiar 
they  were  with  it  [RUDD  98]. 

Goerger’s  second  experiment  mirrored  the  Banker  experiment  in  all  details  except 
that  the  model  was  a  real-time  model  that  allowed  total  freedom  of  movement,  and  that 
the  subjects  were  not  all  active  in  orienteering.  Like  Banker’s  experiments  there  were 
three  groups:  map-only,  real  world,  and  VE.  The  experiment  used  the  same  map  and 
training  area  as  the  Banker  experiment.  After  the  individual's  one  hour  of  training,  they 
were  tested  the  same  as  in  Banker’s  experiment.  Goerger  concluded  from  his  results  that 
the  spatial  ability  of  an  individual  plays  a  “significant  role”  in  the  individual’s  performance. 
He  also  concluded  that  training  conditions  showed  no  significant  effect  on  the  ability  to 
“obtain  and  demonstrate  spatial  knowledge  of  a  natural  environment”  though  the  map 
group  did  perform  better  based  on  observation.  Finally,  he  concluded  that  the  1:5,000 
scale  map  in  the  study  was  so  effective  and  useful  that  it  was  hard  to  beat  [GOER  98b]. 

The  above  studies  allow  us  to  conclude  at  least  some  benefit  to  using  VEs  for  the 
acquisition  of  route  and  landmark  knowledge.  Map’s,  or  other  top-down  representations, 
still  seem  to  be  the  best  at  providing  survey  knowledge.  Ideally,  a  VE  would  combine 
these  advantages,  allowing  the  user  to  either  quickly  use  an  external  map  or  have  one 
incorporated  in  the  model. 

3.  Model  Classifications 

It  quickly  becomes  apparent,  given  the  vast  scope  of  fields  and  tasks  that  are  being 
explored  with  VEs,  that  no  one  set  of  standards  fits  all  models.  Much  as  with  any  real 
world  building,  different  models  must  be  designed  for  their  e3q)ress  purpose  and  have 
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different  features  depending  on  that  purpose.  We  would  expect  a  VE  for  flight  training  not 
to  expend  great  effort  on  modeling  vegetation  appearance  since  it  is  not  the  most  crucial 
part  of  that  task.  In  contrast,  a  VE  designed  for  dismounted  soldiers  has  to  pay  naore 
attention  to  vegetation,  but  less  to  dynamic  physics  of  falling  objects.  Goerger  used  the 
term  “complex  natural  environment”  [GOER  98b]  to  describe  the  areas  of  terrain  conplex 
enough  to  require  detailed  representation  to  portray  them  While  such  a  system  may  not  be 
needed  in  flat  desert  or  plains  areas,  it  becomes  more  desirable  in  other  terrain. 


Figure  2.7.  Fidelity  vs.  Movement  Method 


Most  VEs  can  be  placed  into  one  of  four  broadly  defined  categories  based  on 
terrain  requirements.  These  levels  are  Fixed  Wing  Aircraft,  Rotary  Wing  Aircraft,  Ground 
Vehicles,  and  Dismounted  Movement  [SULL  98].  While  there  may  be  overlap  between 
the  fidelity  and  detail  of  terrain  representation  between  the  four  levels.  Fixed  Wing 
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Aircraft  is  generally  the  least  detailed  with  Dismounted  VEs  being  the  most  detailed 
(Figure  2.7). 

Knowing  the  principal  movement  target  of  the  model  can  help  the  model  designers 
limit  their  work  to  the  correct  scope,  and  not  add  unnecessary  details  or  leave  out  critical 
information.  The  model  used  for  this  thesis  would  fall  into  the  dismounted  movement 
category  [GOER  98b]. 


D.  LAND  NAVIGATION  AND  ORIENTEERING 

1.  Military  Land  Navigation 

Navigation  is  “the  theory  and  practice  of  navigating”  which  means  in  turn,  “to 
make  one’s  way”  or  “to  follow  a  planned  course”  [WEBS  88].  Good  navigation  requires 
knowing  the  current  location,  the  destination,  the  direction  of  travel  (orientation),  and  a 
means  of  travel  [WICK  92].  This  closely  matches  the  military  definition  of  route  planning 
which  is  “knowing  where  you  are,  where  you  are  going,  and  how  you  are  going  to  get 
there”  [ARNG  83].  Navigation  plays  a  critical  role  in  the  military,  and  as  such  an  entire 
manual  devoted  solely  to  the  training  of  map  reading  and  navigation  skills  is  maintained 
(FM21-26  Military  Land  Navigation). 

Army  doctrine  regarding  land  navigation  says  it  is  trained  in  a  building  block 
approach.  First,  basic  map  reading  skills  are  trained,  followed  by  dead  reckoning.  The 
more  efficient,  but  more  difficult  to  learn,  terrain  association  follows.  Later,  soldiers 
develop  route  selection  skills  and  techniques  involving  unit  placement  and  planning  tactical 
movements  [FM21  93].  The  military  focuses  on  two  basic  techniques  for  navigation:  dead 
reckoning  and  terrain  association.  Both  have  advantages  and  disadvantages. 

Dead  reckoning  involves  first  determining  where  a  soldier  is  (based  on  their  map¬ 
reading  skills  and  con^ass  readings),  and  then  determining  the  direction  and  distance  they 
need  to  go.  Soldiers  who  are  dead  reckoning  do  not  deviate  from  their  path  unless  they 
come  upon  an  uncrossable  obstacle.  In  that  case,  they  “box”  around  the  obstacle  and 
quickly  return  to  their  original  route.  The  essential  skills  to  the  dead  reckoning  method  are 
reading  a  con^ass  and  keeping  pace  count  (for  determining  distance).  While  this  method 
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is  easy  to  leam,  it  is  difficult  to  inclement.  Soldiers  need  to  pay  particular  attention  to  the 
con^ass  or  they  may  veer.  This  is  increasingly  difficult  on  long  movements  where  small 
errors  in  the  beginning  can  have  a  dramatic  effect  at  the  end.  Dead  reckoning  is  also  very 
unforgiving.  Without  any  intermediate  terrain  association  or  landmarks,  an  error  is  only 
discovered  once  the  prescribed  distance  is  reached  and  the  soldier  is  not  in  the  right  place. 
Dead  reckoning  can  be  useful  for  short  movements,  in  dense  terrain  (jungle/forest),  in 
featureless  areas  (desert/plain),  or  at  night  when  other  terrain  features  are  not  clearly 
visible.  It  can  also  be  highly  accurate  over  short  distances.  It  has  the  advantage  of  being 
the  easiest  to  leam  and  teach.  Because  of  these  characteristics,  the  Army  advises  soldiers 
to  dead  reckon  a  series  of  short  distances  between  “known”  points  and  locations  [FM21 
93]  to  reduce  the  chance  of  error. 

The  military  prefers  terrain  association  because  it  is  more  forgiving  and  less  time 
consuming  [FM21  93].  The  essential  aspect  of  terrain  association  is  the  ability  to  adjust 
the  route  on  the  way  based  upon  confirmation  of  expected  terrain  features  or  landmarks. 
The  steps  in  terrain  association  consist  of  first  determining  current  location,  and  the 
desired  destination.  The  map  is  examined  for  unique  or  distinctive  feature  of  the  terrain 
that  can  be  used  to  guide  the  movement.  Soldier’s  plan  their  route  to  take  advantage  of 
these  features  to  provide  feedback.  Terrain  association  allows  them  to  take  advantage  of 
handrails,  attack  points,  expanded  objectives,  checkpoints,  catching  features,  and  even 
vegetation  [FM21  93][ARNG  83].  See  Appendix  P  for  definitions  of  these  terms.  To 
properly  terrain  associate,  individuals  should  make  sure  they  keep  the  map  oriented 
properly,  and  they  should  refer  to  it  often.  The  obvious  disadvantage  of  terrain  association 
is  that  the  individual  must  be  able  to  interpret  the  map  and  translate  the  world  around  them 
into  topographic  terms  [FM21  93].  The  ability  to  estimate  distance  and  direction  to  a 
terrain  feature,  and  quickly  estimate  position  accurately  are  difficult  to  teach,  leam,  and 
most  important,  retain.  Often  these  skills  are  only  developed  after  long  hours  of  practice. 

Often  it  is  useful  to  combine  techniques  during  navigation.  The  soldier  may  first 
terrain  associate  to  an  attack  point  (see  Appendix  P  for  definition)  such  as  an  intersection 
or  bridge  close  to  their  desired  objective,  and  then  dead  reckon  firom  there.  Army  doctrine 
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recommends  not  becoming  dependent  on  any  one  method  but  being  capable  of  all  [FM21 
93]. 

The  recent  emergence  of  Global  Positioning  Systems  (GPS)  technology  has  not 
diminished  the  Army  requirements  for  land  navigation  training.  While  GPS  can  prove 
useful  for  determining  location,  the  Army  still  requires  that  soldiers  learn  traditional 
methods.  The  seriousness  that  the  military  places  on  these  skills  can  be  viewed  in  the 
requirement  for  successfully  con^leting  the  land  navigation  course  prior  to  graduation  of 
any  of  the  basic  military  schools  for  officers  and  soldiers  alike.  Indeed,  many  of  the  more 
advanced  schools  have  land  navigation  tasks  that  are  essential  to  pass  and  are  considered 
among  the  most  challenging  requirements  (Ranger  School  and  the  Special  Forces 
Assessment  Selection  course  respectively). 

2.  Sport  Orienteering 

Orienteering  is  con^titive  land  navigation.  The  sport  of  orienteering  was  first 
developed  in  Sweden  in  the  19“*  century  and  was  brought  to  America  in  1946  [FM21  93]. 
The  International  Specification  for  Orienteering  Maps  (ISOM)  describes  orienteering  as  a 
sport  where  runners  complete  a  course  of  control  points  (or  controls)  in  the  shortest  timp-. 
aided  by  map  and  compass  [ISOM  90].  In  fact,  there  are  four  common  types  of 
orienteering  events. 

The  most  common,  and  the  one  that  most  people  think  of  as  orienteering,  is  cross¬ 
country  orienteering.  Here,  numerous  points  are  spread  out  along  the  area.  Contestants 
must  visit  the  points  in  order  but  are  not  constrained  as  to  which  route  they  use  to  get 
there.  Usually  staggered  start  times  are  used.  The  runner  with  the  best  time  who  reaches 
aU.the  controls  in  order  wins. 

A  variation  of  cross-country  is  score  orienteering.  Here  the  control  points  are 
again  spread  out  but  runners  do  not  have  to  visit  them  in  any  order.  Each  control  is  given 
a  numeric  point  value  with  the  farther  controls  worth  more.  The  runner  who  has  collected 
the  most  “points”  after  a  set  time  limit  is  judged  the  winner. 

Route  orienteering  is  more  of  a  training  session.  Here  a  leader  will  lead  the  group 
along  a  route.  The  participants  have  to  draw  the  route  followed  on  their  maps  and 
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annotate  the  locations  of  all  the  controls  they  saw.  At  the  end  of  the  route,  all  the  maps  are 
con:q)ared  to  the  accurate  master  map.  The  closest  map  to  the  master  map  wins. 

Line  orienteering  is  similar  to  route  except  that  there  is  no  leader.  The  participant 
follows  a  route  drawn  on  his  map  to  the  best  of  his  ability  and  annotates  the  locations  of 
all  controls  along  the  route. 

For  the  speed  events  (cross-country  and  score  orienteering)  runners  are  usually 
provided  with  a  clue  sheet  to  assist  them.  This  sheet  provides  data  about  the  terrain 
feature  the  point  is  located  in/on.  The  controls  themselves  are  usually  three-sided  to  be 
recognized  from  any  angle  of  approach. 

Conpetition  orienteering  has  different  levels  of  con^tition.  These  color-coded 
levels  increase  the  distance  and  difficulty  of  the  locations.  White  is  the  easiest,  followed  by 
orange  and  then  red.  It  is  common  for  multiple  courses  to  be  run  over  the  same  area  with 
different  controls  for  the  different  courses  littering  the  same  area.  Unlike  military  land 
navigation,  which  recommends  at  least  300  meters  between  points,  orienteering  has  no 
such  restriction.  In  any  case,  no  two  controls  should  be  within  25  meters  of  each  other  and 
no  two  controls  on  the  same  course  should  be  closer  than  75  meters  unless  they  are  on 
different  terrain  features  [FM21  93]. 

Another  difference  between  most  military  and  orienteering  courses  is  the  map 
itself.  Most  military  navigation  is  done  at  1:50,000  [FM21  93]  or  1:25,000  at  best.  In 
FM21-26,  the  Army  considers  1:50,000  to  be  reasonable  for  orienteering.  Con^etition 
orienteering  is  usually  done  at  1:15,000  scale.  This  is  a  profoimd  difference.  Military 
navigators  are  looking  to  recognize  major  terrain  features.  Sport  orienteerers  are  expected 
to  distinguish  between  minute  differences  and  the  potentially  close  proximity  of  the  points 
adds  to  the  difficulty  of  this  con^etitive  sport. 
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in.  METHODOLOGY 

A.  EXPERIMENT  OVERVIEW 

In  order  to  determine  whether  Virtual  Environments  are  useful  for  transferring 
spatial  awareness  of  natural  regions,  a  series  of  experiments  was  conducted.  This  section 
will  provide  an  overview  of  the  conduct  of  the  experiments,  while  succeeding  sections 
describe  the  tools,  techniques,  and  methodology  in  more  detail.  The  general  sequence  for 
the  experiment  was  the  in-briefing,  battery  tests,  spatial  orientation  test,  model 
familiarization  (VE  Only),  training  in-brief,  training,  pre-course  test  (only  for  subjects  in 
experiment  #1),  actual  course  test,  and  then  debriefing. 

Upon  the  subject’s  arrival,  they  were  read  the  basic  m-brief  by  the  research 
monitor  and  had  to  fill  out  the  initial  consent  forms.  The  in-brief  is  shown  in  Appendix  C. 
The  consent  forms,  including  medical  waiver  are  in  Appendix  D. 

Once  the  subject  had  fiUed  out  the  consent  forms,  the  next  step  was  the  battery  of 
tests.  Subjects  had  to  con^lete  a  test  on  basic  terrain  identification,  as  well  as  a  standard 
medical  color-blindness  test.  Subjects  conq)leted  a  short  informational  questionnaire  that 
asked  where  they  first  learned  to  navigate,  how  long  they  had  been  navigating,  and  at  what 
level  they  would  classify  their  ability.  On  a  separate  page,  subjects  had  to  conq)lete  a  bar- 
line  evaluation  of  their  navigational  ability.  The  redundancy  of  having  the  same 
information  in  two  places  provided  a  confirmation  and  removed  any  bias  that  may  have 
been  associated  with  the  word  descriptions  of  navigational  level  Lastly,  in  this  phase, 
subjects  conpleted  the  Santa  Barbara  sense  of  direction  scale  questionnaire  (see  Appendix 
E).  This  15-question  questionnaire,  developed  by  the  University  of  California  at  Santa 
Barbara,  asks  the  user  to  rate  themselves  on  such  topics  as  ‘T  remember  directions  weD” 
or  “I  always  know  where  I  am”.  This  test  helps  assess  how  confident  the  subject  is  on 
navigational  tasks.  The  map  test,  the  navigation  history  and  bar-line  are  in  Appendix  E. 

The  next  stage  of  the  experiment  was  giving  each  subject  the  Guilford  Zimmerman 
(GZ)  spatial  aptitude  test  [GOER  98b].  This  test  would  determine  a  subject’s  ability  to 
sense  changes  in  direction  and  orientation.  It  was  a  60-question  test,  with  ten  minutes 
allowed  for  completion.  Subjects  had  to  answer  as  many  questions  as  they  could,  but  1/4 
point  was  deducted  for  each  wrong  answer  from  the  total  they  answered  correctly.  No 
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points  were  deducted  for  unanswered  questions.  The  GZ  test  cover  sheet  with  an  example 
question  is  shown  in  Appendix  E.  The  test  was  used  to  segregate  the  subjects  into  high 
and  low  ability  groups.  Because  past  experiments  proved  a  correlation  between  spatial 
ability  and  navigational  performance,  the  goal  was  to  equally  distribute  the  high  and  low 
ability  individuals  into  the  map  and  VE  groups. 

Once  subjects  were  tested,  their  group  was  designated  based  upon  GZ  ability. 
Those  chosen  for  the  VE  groups  were  now  given  15  minutes  of  model  interface 
orientation.  For  this  training,  they  were  put  in  an  environment  that  did  not  resemble  the 
actual  model  area.  This  training  VE  did  have  the  same  interface  controls  and  allowed  the 
user  to  become  familiar  with  the  various  functions  and  capabilities  available  to  them 
through  the  model.  Prior  to  their  completion  of  the  interface  orientation,  they  had  to 
con^lete  an  interface  familiarization  checklist  or,  failing  that,  they  would  be  required  to 
conduct  more  training  at  the  controls.  During  the  course  of  this  familiarization  period,  the 
monitor  would  note  if  the  subject  was  having  any  noticeable  difficulties  with  the  model. 

Now  aU  subjects  were  ready  for  their  training  phase.  First,  they  were  read  a  series 
of  standardized  instructions  based  upon  their  groups  (Map  or  VE)  and  their  experiment 
(Train-to-standard  or  Map  Fidelity).  In  all  cases,  subjects  were  given  a  participant  task  list 
and  important  information  on  marking  their  map  (see  Appendix  G  and  H  respectively). 
This  clearly  laid  out  what  the  subject  was  supposed  to  do  when  they  arrived  at  the  testing 
site.  This  information  helped  focus  the  subjects  during  their  training  on  what  task  they 
were  preparing  for.  For  the  train-to-standard  group,  they  were  also  informed  of  the  pre¬ 
course  test  (see  Section  N.l.  Train-to-standard)  and  the  requirement  that  they  pass  this  at 
the  end  of  their  training  before  they  would  be  taken  out  to  the  actual  course.  The  in-briefe 
for  all  four  of  the  experiment  groups  are  located  in  Appendix  C.  At  this  point  the  monitor 
would  present  the  subjects  their  training  aids,  explaining  each  in  detail.  For  the  train-to- 
standard  experiment  group,  these  aids  included  a  1:5,000  orienteering  map,  an 
orienteering  clue  sheet,  and  side  by  side  photo  comparisons  between  the  control  points  in 
the  real  world  and  in  the  model.  The  map-only  group  photographs  only  showed  the  actual 
points  and  not  those  of  the  model.  For  the  map  fidelity  experiment,  the  1:5,000 
orienteering  map  was  excluded.  In  its  place  was  an  overhead  photograph  of  the  area,  as 
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well  as  a  1:24,000  map  of  the  region.  To  allow  for  easier  reading,  this  map  was  also 
provided  at  1:5,000  scale  but  with  no  additional  information  being  provided  on  the  map. 
These  training  aids  are  discussed  in  more  detail  in  the  following  sections. 

Individuals  would  then  actually  conduct  their  training  based  on  the  standards  of 
the  experiment  they  were  participating  in.  At  the  end  of  the  training  period,  all  subjects 
were  to  have  their  proposed  route  clearly  marked  on  the  map,  showing  the  exact  route 
they  intended  to  follow  on  the  actual  course.  During  the  training  period,  the  monitor 
remained  close  at  hand,  recording  the  time  they  were  taking,  and  answering  any  questions 
that  the  subject  needed  clarified.  For  VE  subjects,  monitors  also  noted  any  interface 
difficulties  and  if  the  subject  appeared  to  be  disoriented  or  lost  in  the  VE.  Upon  the 
successful  con^letion  of  the  training  period  (which  may  have  been  an  hour  or  more),  all  of 
the  training  materials,  except  for  the  clue  sheet,  were  taken  firom  the  subject.  The  monitor 
and  subject  then  left  the  training  room  and  drove  to  Fort  Ord.  Once  they  neared  the  road 
leading  to  the  test  area,  subjects  were  blindfolded  to  ensure  that  they  would  not  arrive  at 
the  testing  area  already  oriented.  This  was  also  done  so  that  they  could  not  see  any  of  the 
control  points  in  route  (two  points  were  directly  visible  from  the  boundary  road  under  the 
right  conditions). 

Once  they  were  at  the  course  location,  subjects  were  again  shown  the  target 
control  point  naarkers  to  remind  them  of  what  they  were  looking  for.  Subjects  donned  the 
GPS  backpack  that  would  record  all  of  their  movements  on  the  course  so  that  any  error 
deviations  could  be  accurately  recorded.  Subjects  were  read  the  Navigation  Course  task 
briefing,  which  outlined  the  exact  task  conditions  and  standards  they  should  follow.  See 
Appendix  C  for  the  briefing.  The  monitor  initially  oriented  the  subjects  using  the  map  and 
conqaass.  The  monitor  also  ensured  that  the  subject  had  their  clue  sheet.  The  monitor 
encouraged  the  subject  to  speak  aloud,  so  that  the  monitor  could  understand  and  record 
what  the  subject  was  thinking.  The  subject  would  have  one  hour  to  complete  aU  the  tasks. 
The  monitor  kept  the  official  time.  At  control  points  two  and  four,  the  monitor  would 
stop  and  have  the  subject  answer  questions.  Time  was  stopped  during  these  periods.  The 
subject  could  also  request  the  con^ass,  the  map,  or  the  map  and  compass  together  at  any 
time.  All  of  these  checks  were  timed  and  recorded.  As  the  subject  traversed  the  course. 
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the  monitor  recorded  their  comments,  and  noted  when  and  where  they  made  errors.  As 
subjects  conpleted  a  task,  the  monitor  would  read  their  next  task  from  the  Navigational 
briefittg  to  ensure  uniformity  across  the  group.  Subjects  would  have  only  one  active  task 
at  any  time.  If  subjects  desired  to  change  their  route,  they  could  do  so.  Once  a  subject 
found  the  control  they  were  looking  for,  the  monitor  would  read  the  next  set  of 
instructions.  If  the  subject  reached  control  point  nine  in  the  one  hour  time  limit  they  were 
given  the  unplanned  route  task  (see  Section  L).  Once  that  task  was  con^lete,  or  upon 
being  timed  out  at  any  portion  of  the  test,  subjects  were  guided  back  to  the  vehicle  by  the 
monitor  where  they  performed  the  whiteboard  test  (see  Section  K.  Whiteboard)  regardless 
of  whether  they  actually  finished  the  course  or  not. 

Once  the  testing  was  finished,  subjects  were  driven  back  to  NPS.  There  they  were 
given  a  detailed  questionnaire  about  the  experiment  and  the  training  tools  they  had  been 
given.  Subjects  were  also  shown  the  route  they  had  traveled  by  downloading  the  GPS  data 
into  ARCVIEW  and  presenting  the  data  plots.  Subjects  were  asked  to  elaborate  on  their 
thoughts  at  the  times  they  made  errors  or  mistakes  during  the  course.  Subjects  were  also 
asked  the  debriefing  questions.  These  questions  are  shown  in  Appendix  E.  Once  this  was 
all  finished,  subjects  were  thanked  for  their  participation  and  reminded  not  to  talk  of  the 
experiment  specifics  to  anyone.  In  total,  the  experiment  took  approximately  four  hours  to 
con^lete.  This  could  be  much  longer  for  the  train-to-standard  experiment. 

B.  TEST  ENVIRONMENT 

The  area  used  for  the  experiment  was  a  1200  meter  by  700  meter  section  of  land 
located  on  what  used  to  be  Fort  Ord  in  the  coastal  area  of  central  California.  The  terrain  in 
that  region  is  characterized  by  shallow  rolling  hills  and  grasslands.  The  area  used  for  the 
course  varied  from  90  meters  to  123  meters  in  elevation  providing  for  distinct  navigational 
terrain  features  yet  not  being  so  extreme  as  to  inqjose  a  physical  challenge  conq)onent  to 
the  experiment.  The  Fort  Ord  region  of  California  has  three  predominant  vegetation  types, 
all  of  which  can  be  found  on  the  course  (see  Appendix  F) 

Oak  Forest  covers  much  of  the  area  (40%).  Inland  and  coast  live  oaks  of  25  to  45 
feet  in  height  are  dominant.  These  trees  have  large  canopies  that  often  touch  their 
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neighbor,  which  can  inhibit  undergrowth  and  allow  for  easy  traversal  though  poorer 
lighting.  Some  tree  canopies  can  extend  almost  to  the  ground  providing  a  visual  as  well  as 
physical  impediment.  Where  the  trees  are  more  scattered  there  is  often  thick  undergrowth 
that  can  slow  travel. 

Another  40  percent  of  the  course  is  covered  by  maritime  chaparral,  a  plant  that 
grows  in  dense  thickets  that  provides  an  extreme  inqjediment  to  cross  country  travel. 
Movement  in  chaparral  is  almost  always  limited  to  the  roads  and  trails  in  the  area  unless 
individuals  plan  to  painfully  “fight”  their  way  through  the  dense  brush.  It  is  possible  to 
traverse  through  Chaparral  in  some  places  but  the  rate  of  travel  is  very  slow  and  the 
density  limits  visibility.  Most  travelers  prefer  to  avoid  these  regions  entirely  or  stick  to 
roads.  No  test  subject  attempted  to  go  through  a  section  of  chaparral  more  than  once. 
Depending  on  the  height  of  the  chaparral,  visibility  ranges  firom  excellent  to  poor.  Often 
travelling  on  a  trail  through  dense  tall  chaparral  is  the  same  as  walking  a  corridor. 

The  third  category  is  perennial  grassland.  It  covers  the  remaining  20  percent  of  the 
course  and  has  knee  high  grasses  with  a  scattering  of  trees  and  patches  of  dense 
,  underbrush.  ExceDent  visibility  and  mobility  characterize  the  region. 

The  testing  area  is  crisscrossed  by  a  series  of  dirt  roads,  trails,  and  footpaths.  Some 
date  back  to  when  the  army  trained  in  the  area,  others  have  come  about  due  to  public 
mountain  biking  and  other  outdoor  recreation.  The  condition  of  these  trails  varies  as  they 
wind  through  the  testing  area,  and  it  is  not  uncommon  for  a  trail  to  be  well  beaten  at  one 
point  only  to  be  overgrown  at  another.  The  area  also  has  numerous  decaying  outhouses, 
shacks,  wooden  pavilions,  telephone  poles  and  other  manmade  structures  that  scattered 
throughout  the  course,  but  usually  located  along  roads  or  trails.  The  number  of  these 
structures  and  trails  provides  ample  opportunity  for  parallel  errors  (see  Section  H. 
ERRORS).  The  area  is  also  dotted  with  old  foxholes,  trench  lines,  pits,  and  cement  pads. 
These  features  can  aid  navigation  but  can  also  induce  parallel  errors. 

Roads  bound  the  testing  area  on  all  four  sides.  The  northern  road  is  paved  but  the 
other  three  are  dirt.  Many  of  the  trails  inside  the  course  connect  with  one  or  more  of  the 
boundary  roads.  Travel  into  the  area  is  prohibited  to  vehicular  traffic  without  special 
approval. 
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C.  MODEL 

The  model  is  a  real  time  replica  of  the  test  environment  and  was  developed  by 
MAJ  Simon  Goerger  [GOER  98b]  based  on  the  aerial  photograph,  the  Banker  course 
map,  and  ground  reconnaissance.  Goerger  built  the  model  using  the  OpenGL  language, 
EasyScene  software.  Coryphaeus,  Inc.  The  model  itself  runs  on  a  Silicon  Graphics 
Industry  (SGI)  Onyx  Infinite  Reality  workstation.  This  machine  has  four  processors 
running  at  194  MHz,  256  Mbytes  of  main  memory,  and  4  Mbytes  of  texture  memory 
[GOER  98b].  Goerger  studied  extensively  the  required  accuracy  and  developed  his  model 
at  one  meter  resolution  [SULL  98]  [GOER  987b].  In  order  to  have  the  model  run  in  real 
time,  Goerger  made  extensive  use  of  level  of  detail  (LOD)  modeling.  This  meant  that  if 
some  objects  were  far  away  they  were  not  drawn,  or  were  drawn  in  a  simpler  manner  to 
save  polygons.  Goerger  discusses  at  length  the  thought  process  he  went  through  in 
deciding  upon  where  exactly  his  transition  points  would  be  [GOER  98b]. 

To  iiqpart  realism  in  the  model,  Goerger  used  digital  photographs  of  trees  and 
bushes  attached  to  billboards  to  populate  the  vegetation  in  his  model.  Four  different  trees, 
three  types  of  bushes,  and  two  types  of  underbrush  were  used  [GOER  98b].  These 
vegetation  billboards  were  not  placed  to  exactly  match  the  individual  trees  found  in  the 
real  world,  but  rather  in  a  representational  manner.  Areas  of  dense  vegetation  in  the  real 
world  would  subsequently  have  more  billboards  in  the  model. 

In  order  to  accurately  place  the  roads,  as  well  as  give  a  sense  of  shadowing, 
Goerger  overlaid  the  terrain  with  the  1993  aerial  photo.  He  then  hand  placed  the  roads  in 
their  appropriate  locations.  All  trails  in  the  model  are  displayed  the  same,  regardless  of 
their  real-world  condition.  This  can  lead  to  the  “yellow-brick  road”  effect  where  subjects 
later  doubted  they  were  on  the  right  trail  because  in  the  model  it  was  clear,  and  in  the  real 
world  it  was  overgrown. 

For  the  man-made  objects,  Goerger  used  digital  photos  and  hand  taken 
measurements  to  accurately  build  and  locate  the  structures  into  his  model.  All  told,  the 
model  included  five  outhouses,  two  shacks,  two  pavilions,  nine  cement  pads,  and  three 
rock  piles.  It  also  had  over  20  individual  telephone  poles,  telephone  fines,  two  trench  fines. 
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and  200  sandbags.  Over  9,000  meters  of  trails/roads  were  included,  as  were  over  14,000 
tree  billboards  [GOER  98b].  Two  military  HMMWVs  were  added  in  an  attempt  to  offer  a 
sense  of  scale  to  the  model. 

The  interface  uses  a  flybox  (see  Figure  3.1)  as  its  interaction  device.  Users  were 
allowed  to  independently  turn  their  head  to  their  right  or  left,  or  look  up  and  down 
without  affecting  direction  of  movement  or  orientation.  The  users  could  designate  a  run  or 
a  walk  mode. 


Figure  3.1.  Flybox  Interface 


The  model’s  avatar  was  set  to  a  standard  height  of  five  feet  eight  inches  above  the 
groimd,  but  users  could  toggle  a  15-meter  pop  up  view  that  enabled  them  to  get  above  the 
trees.  Users  could  also  toggle  an  overhead  “you  are  here”  style  map  that  showed  their 
exact  location  and  direction  of  travel.  The  maximum  speed  allowed  in  the  model  was 
roughly  10  miles  per  hour.  Other  features  in  the  model  were  the  ability  to  instantly  teleport 
to  the  start  or  end  points,  as  well  as  to  any  of  the  control  points.  Users  could  also  get  a 
quick  view  of  any  of  the  control  points  without  actually  moving  there.  Goerger  designed 
the  model  so  that  the  lighting  could  be  adjusted  to  any  one  of  six  levels  (night  to  bright 
daylight) [GOER  98].  Goerger  did  not  implement  any  collision  detection  so  users  could 
unrealistically  move  through  buildings  and  trees.  Other  than  the  15-meter  pop-up  view, 
there  was  no  way  for  users  to  raise  or  lower  the  avatar’s  eye  height.  This  created  some 
difficulties  in  locating  controls  in  the  model,  as  subjects  could  not  lower  their  head  to  see 
under  tree  branches. 
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To  aid  in  navigation  and  to  help  the  user  orient  themselves  in  the  model,  Goerger 
included  a  standard  compass.  This  compass  could  be  used  during  rotation,  but  could  not 
be  used  while  moving.  If  the  user  toggled  the  compass  while  moving,  he  would  stop.  The 
necessity  of  the  conq)ass  in  the  VE  was  evident  given  the  difficulty  of  ensuring  accurate 
body  movements  in  the  VE  (e.g.,  like  turning  around). 

For  the  visual  display,  Goerger  decided  upon  a  three-screen  configuration  with 
103-degree  field  of  view  [GOER  98b].  These  three  40  inch  screens  were  set  up  exactly  67 
inches  fi’om  the  subject  to  ensure  the  103  degrees  was  achieved.  He  felt  that  this  would 
give  the  user  the  best  feeling  of  immersion  and  enable  them  to  use  their  peripheral  vision 
to  maximum  effect.  Without  this  expanded  viewing  area,  users  of  the  model  would  be 
much  more  likely  to  make  parallel  errors  or  be  forced  to  constantly  turn  their  head.  Figure 
3.2  shows  the  three-screen  configuration. 


Figure  3.2.  Three-Screen  Configuration 


In  summary,  based  on  his  extensive  research  into  FOV,  LODs,  and  his  detailed 
measurements,  Goerger  was  able  to  make  an  extremely  accurate  real-time  model  of  the 
Fort  Ord  testing  area.  This  allowed  the  user  to  seamlessly  explore  the  region  both  on  foot 
and  in  the  air  to  quickly  traverse  to  areas  of  interest. 

D.  MAPS 

The  experiments  used  two  different  maps.  MAJ  William  Banker  [BANK  97] 
developed  the  map  used  in  the  first  experiment.  This  was  the  same  map  used  in  both  the 
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F.  DATA  GATHERING  TOOLS 

In  order  to  measure  whether  subjects  were  following  their  planned  path  within  the 
margin  of  error  a  differential  global  positioning  system  (DGPS)  backpack  was  used.  This 
system  would  take  location  information  transmitted  from  satellite  and  record  it  in  a 
Newton  MessagePad  130  (see  Figure  3.3).  These  data  plots  were  taken  every  five 
seconds.  Once  the  experiment  was  finished,  the  location  plots  were  downloaded  from  the 
MessagePad  into  a  PC.  Using  FieldWorker  software,  the  data  was  overlaid  onto  an 
overhead  photo  of  the  region.  This  enabled  the  monitor  to  accurately  measure  exactly  how 
far  off  route  subjects  had  traveled  during  errors.  This  program  was  also  mvaluable  during 
the  out-briefing  for  showing  the  individual  exactly  what  they  had  done,  and  for  discussing 
exactly  where  they  made  mistakes. 


Figure  3.3.  DGPS  Backpack  and  MessagePad  130  [GOER  98b] 

A  second  means  of  data  collection  was  the  monitor  himself.  He  would  record  the 
actions  of  the  subject  as  they  progressed  through  the  testing  environment.  All  map-checks 
aid  route  changes  were  recorded,  as  was  the  time  it  took  individuals  to  arrive  at  control 
points.  The  monitor  also  recorded  the  exact  time  that  subjects  left  their  planned  route, 
where  the  error  occurred,  and  the  type  of  error.  This  was  needed  in  case  of  DGPS  failure. 
Subjects  were  encouraged  to  “think  out  loud”  (Appendix  K)  and  the  monitor  recorded 
their  comments  and  actions. 
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G.  COURSE 

The  testing  course  itself  was  design  and  set-up  by  MAJ  William  Banker 
[BANK97].  He  used  the  International  Orienteering  Federation’s  specifications  [ISOM  90] 
to  develop  the  course  and  the  course  map  (Appendix  F).  In  all  but  one  respect,  the  course 
conforms  to  thee  standards  of  an  orienteering  “orange”  course  [ISOM  90].  The  only 
deviation  firom  the  standard  is  the  level  of  map  detail.  Orienteering  con^tition  normally 
uses  a  1:15,000  scale  map.  The  official  specification  allows  the  use  of  1:10,000  in 
exceptional  cases.  MAJ  Banker’s  map  (Appendix  F)  is  of  1:5000  scale  (see  Section  B. 
MAPS).  The  other  map  provided  was  a  1:24,000  scale  map  that  was  not  detailed  enough 
by  sport  orienteering  standards.  However,  the  military  considers  1:50,000  to  be  the 
standard  navigation  map  and  they  seldom  see  anything  better  than  1:25,000  [FM21  93]. 
The  course  greatly  differs  firom  standard  military  land  navigation  courses  in  that  the 
control  points  are  not  at  least  300  meters  apart  and  the  overall  course  is  not  over  2500 
meters  in  length. 

The  start  point  for  the  course  is  the  intersection  of  Gigling  and  Watkins  roads  on 
old  Fort  Ord.  The  course  consists  of  the  start  point  and  nine  control  points  which  must  be 
found  in  order.  If  a  point  is  discovered  out  of  order,  no  credit  is  given  until  the  point  is 
found  again  is  the  proper  sequence.  The  points  were  placed  so  that  there  were  numerous 
routes  to  the  controls.  No  controls  were  located  directly  on  a  road  or  trail  as  per  ISOM 
standards.  Because  of  the  layout  of  the  course,  it  was  not  efficient  to  reuse  previously 
traveled  routes  during  the  conduct  of  the  course.  Three  points  were  located  below  ground 
level.  Control  point  two  was  in  a  narrow  pit,  point  four  in  a  wide  shallow  depression,  and 
control  seven  in  a  dry  ditch.  No  points  were  hidden  in  dense  brush  or  inside  man-made 
structure.  All  controls  were  located  near  identifiable  terrain  features  or  objects.  The  first 
three  controls  in  the  course  are  located  relatively  close  together.  This  often  allows  the 
subjects  to  gain  a  sense  of  confidence.  Direct  line  distance  between  the  points  on  the 
course  is  just  over  2000  meters,  though  a  route  following  only  roads  and  trails  could  easily 
double  this. 
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H.  ERRORS 


There  are  five  different  classifications  of  errors  that  individuals  commit  during  the 
course.  Again,  as  noted  earlier,  individuals  are  assessed  an  error  if  they  travel  more  than 
five  meters  off  of  their  designated  path  (if  path  was  planned  on  a  trail/road)  or  if  more  than 
15  meters  off  of  a  cross-country  route.  Once  an  individual  is  assessed  an  error,  the 
distance  he  travels  is  calculated  until  he  returns  to  his  preplanned  route,  marks  a  new 
route,  finds  the  control  point,  or  his  one  hour  time  limit  for  the  experiment  expires. 
Individuals  can  commit  multiple  errors  if  they  move  off  their  route,  correct  themselves, 
and  move  off  their  route  again.  In  addition,  if  they  mark  a  new  route  and  then  veer  pff  that 
they  also  get  multiple  errors.  Each  of  these  errors  was  recorded  and  weighted  differently 
as  noted  in  the  following  discussion. 

By  far  the  most  common  type  of  error  encountered  was  a  mirror  error.  This  is 
where  the  subject,  upon  reaching  a  decision  point  (a  branch  in  the  trail  for  instance) 
chooses  to  follow  the  incorrect  one.  Given  the  numerous  intersecting  trails  and  roads  in 
the  course,  it  was  quite  common  for  individuals  to  make  this  mistake.  Given  the  high 
potential  for  this  t5q)e  of  error,  it  was  given  a  basic  weight.  Figure  3.4  shows  a  mirror 


Figure  3.4.  Mirror  Error  Exan:5)le 


Another  common  type  of  error  is  a  parallel  error.  This  occurs  when  the  individual 
thinks  he  is  at  one  location  but  is  actually  at  another.  The  subject  may  be  looking  for  a 
building  next  to  an  intersection  and  find  a  different  location  with  the  same  features.  The 
subjects  mistakenly  thinks  he  is  at  his  desired  location.This  course  has  numerous  similar 
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trail  intersections,  buildings,  and  locations  that  make  this  type  of  error  common  (see 
Section  A.  EXPERIMENT  OVERVIEW). 


A  compound  error  occurs  when  individuals  make  a  mirror  or  parallel  error  off  a 
newly  planned  route.  This  can  occur  after  a  reorientation  error  (see  later),  or  if  an 
individual  himself  requests  a  new  route  and  then  errs  oflf  it. 

An  out-of-bounds  error  occurs  when  a  subject  leaves  the  course  boundaries.  W^e 
there  is  no  restriction  against  using  the  boundary  roads,  should  an  individual  take  even  one 
step  oflf  the  wrong  side  of  a  boundary  road  he  is  assessed  an  error.  The  individual  is  not 
oriented  as  to  where  he  is  on  the  map  if  he  makes  this  eiror.  He  is  only  informed  that  he 
has  left  the  course.  There  is  no  error  distance  recorded  with  this  type  of  error.  Because 
being  told  you  left  the  course  boundary  provides  some  information  and  since  error 
distance  is  not  calculated,  this  type  of  error  is  tracked  under  map-checks. 

Should  a  subject  leave  their  planned  route  for  more  than  15  continuous  minutes 
and  not  be  making  progress  towards  their  next  control  point,  they  will  be  reoriented  as  per 
Navigation  Course  task  briefing  (Appendix  C)  and  be  assessed  a  reorientation  error.  At 
this  tune  the  subject  will  be  shown  their  location  on  the  map,  and  given  one  minute  to  plan 
a  new  route  to  their  next  control  point.  Since  the  subject  uses  the  map  and  is  shown  their 
exact  location,  this  type  of  error  is  recorded  under  map-checks  and  not  distance  errors.  No 
error  distance  is  recorded  for  this  type  of  error  since  individuals  may  make  con^)ound 
errors  later,  which  would  result  in  the  double  counting  of  error  distance. 

Note  that  when  individuals  made  an  error,  only  the  initial  cause  of  the  error  was 
recorded  not  any  subsequent  misjudgments  they  may  noake  during  the  course  of  the  error. 
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Thus  if  a  person  makes  a  parallel  error  and  thinks  he  is  at  a  different  location,  and  he  takes 
a  wrong  trail  based  on  that,  it  will  be  a  parallel  error  and  not  a  mirror  error.  Also,  if  a 
subject,  after  taking  a  wrong  trail  (mirror  error)  then  assumes  they  are  at  a  different 
location  (parallel  error)  and  make  decisions  based  on  this  erroneous  conclusion,  it  is  still 
recorded  as  a  mirror  error.  Once  an  individual  has  made  an  error,  error  distance  continues 
until:  the  subject  returns  to  their  planned  route  for  the  control  point  they  are  looking  for 
(at  any  point  along  the  route),  the  subject  finds  the  control  point,  the  subject  creates  a  new 
route  (though  they  may  start  a  new  error  if  they  plan  this  new  route  not  Jfrom  their  actual 
location)  or  the  experiment’s  one-hour  time  limit  expires. 

I.  SUBJECTS 

The  subjects  for  this  experiment  consisted  of  20  individuals  ranging  in  age  firom  29 
to  40  with  an  average  of  33.  All  were  active  duty  Army  (11)  or  Marine  (9)  officers  of 
either  Captain  (17)  or  Major  (3)  rank.  75%  of  them  had  ten  or  more  years  of  experience 
navigating,  with  the  rest  having  five  or  more  years.  95%  of  the  subjects  rated  above 
average  on  the  Santa-Barbara  sense  of  direction  scale.  Most  self-assessed  their  ability  as 
intermediate,  with  several  claiming  expert.  None  felt  they  were  beginners.  However,  when 
evaluating  themselves  using  the  bar-line,  two  subjects  did  place  themselves  in  the  beginner 
category.  None  scored  lower  than  17  out  of  20  on  the  map  reading  test.  Only  one  was 
color-blind.  Two  subjects  were  female.  All  the  subjects  had  no  prior  knowledge  of  the 
testing  area,  nor  had  they  any  experience  with  the  maps  used  or  the  virtual  environment. 
There  was  no  tangible  con:q)ensation  for  the  subjects.  Data  was  collected  firom  22  April 
1999  to  23  July  1999.  The  subjects  were  divided  into  four  groups:  train-to-standard  VE 
(VEl),  train-to-standard  map-only  (MAPI),  low-resolution-map  VE  (VE2),  low- 
resolution-map  map-only  (MAP2). 

J.  WHEEL  TEST 

The  wheel  test  was  used  twice  during  the  conduct  of  the  actual  course  as  a  means 
of  determining  if  the  subjects  acquired  survey  knowledge.  The  tests  were  taken  at  control 
points  two  and  four.  At  both  points,  subjects  were  asked  to  point  towards  one  location 
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they  had  already  visited  and  two  they  had  not.  In  no  instances  were  these  locations  the 
ones  just  previous  or  next  in  line.  At  control  point  two  subjects  point  to  the  start  point, 
control  point  five,  and  control  point  nine.  At  control  point  four  they  point  to  control  point 
one,  six,  and  eight.  The  time  it  takes  the  subject  to  con^lete  this  task  is  recorded.  Also 
what  direction  they  face  is  recorded.  Once  the  subject  finishes,  the  results  are 
photographed  and  measured  on  the  spot.  During  this  procedure,  the  official  clock  is 
stopped  so  that  the  individual  is  under  no  time  constraint  pressure.  The  actual  wheel 
consisted  of  a  12  x  12  sheet  of  laminated  cardboard  with  a  seven-inch  full  color  wheel  (see 
Figure  3.6)  and  three  numbered  arrows.  The  wheel  had  16  colored  segments.  Colored 
segments  were  used  so  that  the  individual  would  not  confuse  the  wheel  with  a  con^ass 
[GOER  98b].  Individuals  were  allowed  to  orient  anyway  that  they  choose  during  the  test. 


Figure  3.6.  Wheel  Test 

K.  WHITEBOARD  TEST 

At  the  completion  of  the  course,  all  subjects  are  given  the  white  board  test 
regardless  of  whether  the  found  all  the  control  points  or  not.  Like  the  wheel  test  above, 
this  test  evaluated  the  subject’s  survey  knowledge  of  the  course  area,  but  in  a  more 
exocentric  manner.  This  test  consists  of  a  blank  white  board  and  ten  colored  chits 
representing  the  control  points.  Subjects  are  instructed  to  lay  the  chits  out  in  spacing  and 
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orientation  to  match  the  real  world  layout  of  the  course.  Figure  3.7  shows  a  subject  doing 
the  whiteboard  test.  This  layout  is  photographed  and  then  analyzed  later  to  determine  how 
far  oflF  from  the  real  world  it  is. 


Figure  3.7.  WhiteBoard  Test 


L.  UNPLANNED  ROUTE  TEST 

Upon  completion  of  the  course,  subjects  who  successfully  found  all  nine  of  the 
control  points  would  have  their  survey  knowledge  tested  again.  They  would  be  asked  to 
point  to  control  point  four,  and  how  far  they  thought  they  were  from  it.  They  would  be 
asked  to  describe  how  they  would  get  back  to  control  point  four  if  they  had  to  go  there 
again.  After  describing  the  route,  subjects  would  be  given  ten  minutes  to  follow  the  route 
they  had  described.  The  monitor  would  record  if  they  followed  the  route  that  they 
described,  what  errors  (if  any)  they  made,  and  if  they  found  the  control  point  in  the  ten 
minute  time  Umit.  If  the  subject  made  an  error,  the  cause  of  the  error  and  the  error 
distance  would  be  recorded.  This  data  would  later  be  analyzed  to  see  how  well  they  knew 
their  environment  and  could  they  conduct  an  unplanned  movement  without  aid  of  map  or 
compass. 

M.  RESEARCH  MONITOR 

The  research  monitor  followed  the  text  in  the  experimental  outline  to  the  letter  to 
ensure  uniformity  of  test  experimental  conditions  to  all  subjects.  During  the  train-up 
period,  the  monitor  observed  the  subjects  to  see  how  they  conducted  their  time.  He  was 
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on  hand  to  answer  specific  questions  about  the  course,  record  where  individuals  had 
difficulty  during  the  train-up  time,  as  well  as  administer  the  pre-course  test. 

The  purpose  of  the  research  monitor  during  the  course  was  to  record  the 
comments  and  actions  of  the  test  subject  (in  addition  to  the  tracking  tools)  while 
remaining  in  the  background.  This  meant  prompting  the  subject  for  their  thoughts  but  not 
distracting  them  firom  their  task.  The  monitor  kept  the  official  time,  which  he  stopped  only 
when  recording  information  or  asking  the  subject  questions.  The  monitor  also  needed  to 
record  when  errors,  map  checks,  and  route  changes  occurred,  and  to  ask  the  in-course 
questions  (see  wheel  test,  whiteboard  test,  and  unplanned  route  test).  The  monitor 
recorded  the  time  the  subject  arrived  at  the  various  control  points  and  had  to  be  prepared 
to  orient  the  individual  as  necessary  (see  re-orientation  error).  Finally,  the  monitor  carried 
all  the  water  and  supplies  other  than  the  clue  sheet,  which  was  carried  by  the  subjects. 

N.  TRAINING 

1.  Experiment  #  1:  Train-to-Standard 

In  his  conclusions,  Goerger  speculated  that  the  limited  training  time  may  have  HaH 
an  effect  on  the  outcome  [GOER  98b].  Based  on  his  data  he  felt  VE  subjects  who  actually 
performed  multiple  runs  through  the  simulated  environment  did  better,  at  least  in  terms  of 
finishing  the  course.  In  addition  to  the  lack  of  iterations,  Goerger  also  felt  that  the  short 
training  time  may  not  have  allowed  VE  subjects  to  both  explore  the  environment,  practice 
potential  routes,  and  build  a  valid  mental  map  of  the  environment  all  in  the  span  of  one 
hour  [GOER  98b].  He  envisioned  that  as  exposure  time  increases,  the  performance  of 
both  the  VE  and  real-world  groups  would  exceed  that  of  the  map-only  group.  Based  upon 
these  conjectures,  the  training  portion  of  the  experiment  was  ripe  for  variation. 

Educational  studies  involving  VEs  have  shown  that  students  learn  best  when 
“multiple  practice”  is  included.  In  this  case  multiple  practice  includes  not  only  “repeated 
practice  of  a  topic”  but  also  “practice  firom  multiple  representations”[MCLE  96]. 

The  Army  places  heavy  emphasis  on  training.  It  is  how  they  prepare  for  war  and 
the  commonly  repeated  mantra  is  “Our  top  priority  is  training”  [FM25  88].  Army 
doctrine  demands  that  training  be  performance-oriented,  hands-on,  and  challenging.  Since 
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training  is  so  important  to  the  Army,  it  requires  that  all  training  events  have  tasks, 
conditions,  and  standards.  The  Army  feels  soldiers  perform  better  when  given  these 
guidelines  [FM25  90].  The  tasks  are  the  concepts  to  be  trained.  The  conditions  are  the 
tools  that  can  be  used  and  the  surrounding  area.  The  standards  are  what  the  soldier  must 
be  able  to  do  in  order  to  be  considered  “trained.” 

FM25-100,  Training  the  Force  describes  the  Army’s  view  of  training  to  time 
versus  training  to  standard: 

Demand  training  standards  are  achieved.  Leaders  anticipate  that  all  tasks 
will  not  be  performed  to  standard.  Therefore  design  time  into  training 
events  to  allow  additional  training  on  tasks  not  performed  to  standard.  It  is 
more  inqjortant,  however,  that  they  achieve  the  established  standard  on  a 
limited  number  of  tasks  during  a  training  event  than  to  attempt  many  and 
fail  to  achieve  the  standards  on  any  [FM25  88]. 

The  Army  also  says  training  becomes  more  effective  when  it  is  standard  based  than  when 
it  is  time  based  or  procedure  based  [FM25  90]. 

When  designing  the  original  experiment.  Banker  arbitrarily  chose  the  one-hour 
training  time.  No  analysis  was  done  to  determine  if  this  was  an  optimal  time,  or  even  if  it 
was  sufficient.  The  Goerger  experiment  continued  the  use  of  this  arbitrary  time  limit  to 
training.  Based  upon  Goerger’s  suspicions,  as  well  as  the  supporting  research  from  other 
VE  studies,  and  most  importantly,  from  the  Army  philosophy  towards  training,  the  first 
experiment  did  away  with  the  arbitrary  time  limit  to  training  and  instead  incorporated  a 
standard  based  system. 

Having  decided  to  implement  a  standard  based  system,  it  was  necessary  to  decide 
exactly  what  the  standard  should  be.  Because  the  experiment  used  two  different  group 
with  two  entirely  different  media  (maps-only  vs.  VE)  it  was  apparent  that  the  standard  for 
each  group  would  be  slightly  different,  yet  had  to  be  as  similar  as  possible.  The  first  step  in 
this  area  was  determining  what  exactly  the  experiment  was  testing.  While  the  context  of 
the  experiment  would  be  looking  at  survey  knowledge  at  many  places,  the  most  in^ortant 
aspect  to  completing  the  course  was  developing  the  landmark  and  route  knowledge.  Thus, 
the  e;q)eriment  needed  a  standard  that  tested  this  learning. 
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For  each  group,  a  pre-course  test  was  developed.  When  the  subject  was  read  their 
training  instructions  they  would  be  informed  of  the  pre-course  test  and  told  its  standards 
from  the  start.  They  were  informed  of  the  requirement  to  pass  this  pre-course  test  in  order 
to  finish  their  training  period.  There  would  be  no  time  limit  to  their  training.  When  they 
felt  they  were  ready,  they  would  attempt  the  pre-course  test.  If  they  passed  to  standard 
then  their  training  would  end  and  they  would  proceed  out  to  the  course.  If  they  did  not 
pass  the  test  to  standard  the  pre-course  test  would  be  stopped  and  they  would  be  told 
why/how  they  failed.  They  would  then  be  given  the  opportunity  to  conduct  more  training 
before  attenpting  the  test  again.  There  would  be  no  limit  on  the  number  of  times  the 
person  could  potentially  test. 

For  the  VE  group,  the  design  of  the  pre-course  test  was  straightforward.  The 
subject  would  be  forced  to  run  their  selected  route  in  the  VE  and  actually  prove  that  they 
could  find  aU  the  points  (at  least  in  the  VE).  They  had  to  be  aUowed  access  to  their  map 
during  this  training  period,  as  map  checks  were  available  out  on  the  course.  However,  the 
experiment  didn  t  want  individuals  to  run  the  test  with  map  in  hand,  so  they  were  limited 
to  two  thirty-second  map  checks.  As  in  the  real  experiment,  they  would  not  be  allowed  to 
move  during  a  map  check.  During  the  pre-course  test,  the  subjects  were  also  not  allowed 
to  use  the  15  meter-overhead  view,  the  top-down  view,  nor  the  teleportation  features  of 
the  VE.  These,  while  potentially  useful  for  training,  would  not  be  available  in  the  actual 
course  so  were  prohibited  to  ensure  that  the  subjects  were  not  reliant  on  them.  Due  to  the 
difficulties  of  determining  exact  orientation  in  a  VE,  the  use  of  the  compass  could  not  be 
prohibited  during  the  pre-course  test.  However,  subjects  were  informed  that  their  usage 
was  monitored  and  if  it  were  found  that  they  were  overusing  the  compass,  they  would  be 
failed.  Overuse  in  this  case  was  regarded  as  constantly  stopping  to  check  direction  of 
movement.  All  compass  checks  during  the  pre-course  test  were  recorded  as  well. 

As  it  was  possible  for  people  to  get  lost  and  recover  in  the  actual  course,  this 
possibility  was  incorporated  into  the  pre-course  test.  If  subjects  left  their  designated  route 
(based  on  monitor  observation)  they  were  assessed  an  error.  After  five  minutes  if  they 
not  corrected  themselves,  then  they  were  stopped  and  oriented  by  the  monitor.  Subjects 
would  be  allowed  to  be  oriented  twice.  If  they  required  a  third  orientation  they  were 
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considered  to  have  failed.  In  addition,  if  subjects  left  the  course  boundaries,  they  failed. 
Note  that  due  to  the  inability  to  actually  see  through  the  branches  of  the  trees  in  the 
model,  individuals  actively  searching  for  a  point  in  the  correct  vicinity  were  not  assessed 
to  be  in  error.  The  pre-course  briefing  for  the  group  is  shown  in  Appendix  C. 

For  the  map-only  group,  devising  the  pre-course  test  was  more  difficult.  Clearly, 
the  subject  had  to  show  that  they  in  fact  had  a  clear  understanding  of  where  they  wanted 
to  go  and  how  they  would  travel  between  the  points,  just  as  the  VE  group  had  to  do.  The 
map  group,  however,  lacked  the  simulator  with  which  to  demonstrate  this  knowledge.  The 
best  way  for  the  subjects  to  pass  along  this  information  was  to  verbally  talk  their  way 
through  the  route  they  would  follow.  Appendix  C  shows  the  instructions  the  Map  group 
was  given.  As  with  the  VE  group,  the  map-only  group  was  given  two  map  checks  so  that 
they  could  refi-esh  their  memory  of  their  route  if  necessary.  The  monitor  encouraged  the 
subjects  to  be  as  detailed  as  possible  on  how  they  would  travel,  but  did  not  ever  define  a 
necessary  style  for  fear  of  biasing  the  results.  For  instance,  if  the  instructions  required  the 
subject  to  mention  distance,  then  that  may  have  perhaps  changed  their  planning.  Thus  no 
examples  were  given  to  the  subjects;  they  were  just  encouraged  to  be  as  detailed  as  they 
could  be.  Thus,  it  was  left  to  the  monitor’s  judgement  if  the  subject  had  a  clear  picture  in 
his  mind  of  how  he  would  travel  between  the  points.  If  the  subject  was  vague  or  described 
a  route  other  than  what  they  showed  on  their  plan,  the  monitor  would  ask  them  to 
describe  the  route  segment  again,  preferably  in  more  detail.  More  than  two  errors  of  this 
type  would  result  in  failing  the  pre-course  test  and  return  to  the  study  time.  It  is  important 
to  note  that  while  the  monitor  would  ask  for  clarification  if  the  subject  described  a  wrong 
turn,  the  monitor  did  not  notify  subjects  if  they  had  misidentified  a  terrain  feature  on  the 
map  (i.e.,  a  road  as  a  stream,  or  a  depression  as  a  hill).  This  was  done  so  as  not  to  remove 
on  of  the  potential  hazards  of  reading  a  map,  the  misidentification  of  a  feature.  This  was 
felt  to  be  fair,  as  it  was  also  possible  for  the  VE  group  not  to  correctly  associate  the 
terrain  or  to  misidentify  a  landmark  and  stiU  find  the  points  [GOER  98b].  The  test  was  to 
determine  if  the  subject  had  a  clear  picture  in  his  mind  as  to  how  they  planned  to  conduct 
the  test,  not  a  guarantee  that  it  was  correct  or  would  be  successful. 
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2.  Experiment  #  2:  Varying  Map  Fidelity 

For  the  second  experiment,  it  was  decided  to  go  back  a  focus  on  the  map  given  to 
the  test  subjects.  As  explained  earlier  in  Section  D.  MAPS,  Banker  designed  a  1:5,000 
scale  orienteering  map  for  the  experiment  which  Goerger  also  used  [BANK  97]  [GOER 
98b].  Goerger  had  suspected  that  the  fidelity  of  this  map  might  have  skewed  the 
performance  of  the  subjects  in  his  experiment  [GOER  98b].  He  estimated  that 
performance  would  decrease  for  a  map-only  group  based  on  map  scale.  Figure  3.8 
visualizes  Goerger’ s  supposition. 


- Real  Time  VEs 

- Non  Real  Time  VEs 

“  *  -  1:5,000  Map 
—  -  -1:24,000  Map 
-  1:50,000  Map 


Figure  3.8.  Goerger’s  Performance  vs.  Map  Resolution  Theory 


Remember  that  the  Army  considers  1:50,000  to  be  the  standard  and  sufficient  scale 
for  land  navigation,  with  1:25,000  being  the  absolute  best  available  for  training  and 
operations  [FM21  93].  Also,  keep  in  mind  that  ISOM  states  that  maps  should  be  at 
1:15,000  scale  for  competition,  with  1:10,000  scale  maps  being  allowed  in  exceptional 
cases  and  requiring  official  approval  [ISOM  90].  In  all  cases,  Goerger’s  subjects  were 
mihtary  officers  or  civilians  with  orienteering  experience.  This  group  was  either  familiar 
with  navigating  given  1:15,000  orienteering  maps  or  with  1:50,000/1:25,000  topographic 


48 


maps.  The  subjects,  when  presented  with  a  greater  detail  of  information  and  in  a  format 
they  were  used  to,  may  have  been  able  to  gain  an  advantage  based  on  the  level  of  detail 
alone.  This  is  based,  in  part,  on  the  observation  of  real  world  and  VE  subjects  using  part 
of  their  time  to  explore  possible  routes,  not  merely  memorizing  the  selected  route.  Map- 
only  subjects  seemed  to  quickly  decide  upon  a  route  and  then  spend  their  time  memorizing 
it  [GOER  98b].  This  can  also  be  observed  in  the  after  action  questionnaire  results.  The 
subjects  rated  the  map  as  more  than  adequate  with  it  averaging  a  4  out  of  5. 

In  order  to  determine  if  the  scale  of  the  map  had  an  in5)act  on  the  results,  the 
experiment  went  back  to  the  methodology  of  the  original  Goerger  experiment  [GOER 
98b]  (see  Chapter  H).  While  the  same  methodology  was  used,  the  map  itself  was  changed. 
Instead  of  the  1:5,(X)0  map  created  by  Banker,  subjects  were  given  a  more  realistically 
scaled  two-sided  map  sheet.  On  one  side  the  map  sheet  was  a  to  scale  1:24,000  map  of  the 
course  taken  from  the  commercially  available  1:24,000  scale  map  of  the  Fort  Ord/Seaside 
California  region.  This  map  was  purchased  from  the  National  Geological  Department.  Its 
basic  terrain  data  was  surveyed  in  1929.  It  was  last  updated  in  1983.  On  the  flip  side  of  the 
map-sheet  was  this  same  map  blown  up  to  1:5,000  scale.  However,  no  additional 
information  was  added.  In  effect,  it  just  made  the  map  easier  to  read.  While  this  showed 
the  same  basic  terrain  as  the  Banker  map,  there  were  some  significant  differences.  First, 
the  1:24, 0(X)  map  showed  one  kind  of  vegetation,  conq)ared  to  the  seven  different  types 
on  the  Banker  Map.  In  addition,  the  trail  network  shown  on  the  new  map  was  nowhere 
near  as  extensive  as  the  Banker  map.  One  thing  the  new  map  did  make  much  clearer, 
however,  was  the  depression  surrounding  control  point  four.  This  had  been  difficult  to 
discern  on  the  Banker  map,  but  was  readily  visible  on  the  new  map. 

The  second  training  aid  added  was  the  overhead  photo  of  the  region.  This  photo 
was  used  in  the  VE  model  where  it  provided  shading  in  the  top-down  capability  [GOER 
98b].  In  order  to  ensure  fairness,  this  information  had  to  also  be  provided  to  the  map 
group.  This  is  not  entirely  unrealistic  either.  For  most  important  missions,  units  will  be 
provided  with  aerial  photographs  of  the  objective  area,  even  if  not  recent.  This  photograph 
w^  taken  circa  1993  and  the  area  had  had  minor  changes  to  the  trail  network  from  what 
was  shown  in  the  photo.  WHiile  the  control  points  were  shown  on  the  photo,  no  legend 
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was  provided  concerning  vegetation  or  distance.  While  the  photo  was  close  to  1:5,000 
scale  it  was  not  an  exact  match,  so  subjects  could  not  directly  transfer  distance 
measurements  from  the  map  to  the  photo  and  vice  versa.  Neither  the  new  map  nor  the 
photo  showed  any  of  the  mad-made  structures  that  had  been  so  clearly  shown  in  the 
Banker  map.  They  also  did  not  show  the  breaks  in  the  vegetation,  the  tree  lines,  nor  the 
small  depressions,  pits,  trenches,  or  rock  piles. 

As  mention  earlier  in  this  section  the  conduct  of  the  experiment  was  the  same  as 
the  Goerger  experiment.  The  exact  wording  of  the  various  experiment  sections  are  found 
in  Appendix  C.  In  addition  to  the  map  changes,  one  additional  change  was  made  in  the 
conduct  of  the  experiment.  Because  of  the  inclusion  of  two  different  training  aids  for  the 
map  (the  map  and  the  photo),  the  time  allowed  for  a  map  check  and  for  the  map  and 
compass  check  together  was  increased.  For  this  experiment  map  checks  were  given  one 
minute,  while  map  and  compass  checks  together  were  given  one  minute  and  thirty 
seconds.  This  extra  time  was  allocated  to  allow  the  individuals  to  peruse  both  training  aids 
during  any  map  checks.  Other  than  the  change  in  training  aides  and  the  difference  in  the 
time  for  the  map  checks,  all  other  aspects  of  the  training  period  and  testing  were  the  same 
as  in  both  the  Banker  and  Goerger  experiments. 
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IV.  ANALYSIS 


A.  RESULTS 

1.  General  Information 

The  two  different  experiments  used  in  this  thesis  were  designed  to  test  the  two 
hypotheses  regarding  the  spatial  knowledge  gained  by  participants  exposed  to  different 
training  methods.  Participant’s  overall  performance  was  evaluated  based  on  the  data 
acquired  during  their  conduct  of  the  actual  orienteering  course.  This  data  comes  from  the 
results  of  the  participant’s  performance  on  the  landmark,  route,  and  survey  tasks. 

a.  First  Experiment  Hypothesis  (Train-to-Standard): 

Given  extended  exposure  to  training  materials  and  the  requirement  to  meet 
a  set  standard  before  continuing,  individuals  with  access  to  a  real-time  virtual  environment 
will  outperform  those  who  are  only  exposed  to  a  map  and  photos  of  the  control  points 
with  a  similar  standard. 

b.  Second  Experiment  Hypothesis  (Map  Fidelity): 

Given  an  hour  exposure  to  training  materials  to  include  a  1:24,000  scale 
map  of  the  region,  individuals  with  access  to  the  VE  will  perform  less  poorly  conpared  to 
their  counterparts  with  a  1:5,000  map  than  those  of  the  map-only  group  to  their 
counterparts  with  the  same  conditions. 

2.  Power  Analysis 

The  results  of  the  experiments  are  presented  as  box  plots  with  the  mean,  standard 
deviation,  and  standard  error  shown.  Often  extreme  outlying  data  is  depicted  as  dots  on 
these  plots.  Most  of  the  tests  conducted  on  the  data  were  two  way  analysis  of  variances 
(ANOVA).  The  primary  analysis  was  based  on  the  training  group,  but  a  host  of  possible 
factors  were  analyzed  and  are  discussed  in  Section  B.  The  sample  size  of  this  experiment 
combined  with  the  Goerger  experiment  [GOER  98b]  is  35  subjects  in  seven  different 
groups.  An  a  value  of  0.05  was  used  to  determine  significance.  The  resulting  power  value 
(1-p)  was  0.3051.  Due  to  the  high  number  of  degrees  of  freedom  and  relatively  low 
sample  size,  it  may  be  unwise  to  draw  conclusions  based  solely  on  the  failure  to  determine 
a  positive  effect.  Note  that  as  with  the  Goerger  experiment  [GOER  98b],  the  simultaneity 
of  effects  was  not  considered  so  only  ANOVA  tests  were  conducted.  This  is  not  to  say 
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that  there  are  no  multiple  factor  effects,  just  that  this  was  not  examined  as  part  of  this 
thesis.  Future  work  may  wish  to  reexamine  the  data  using  multiple  analyses  of  variance 
(MANOVA). 

Note  that  for  the  charts  shown,  the  following  symbology  is  used:  MAP  is  used  to 
represent  those  subjects  who  were  map-only  in  the  original  Goerger  experiment.  MAPI  is 
those  subjects  who  were  map-only  in  the  train-to-standard  experiment.  MAP2  is  those 
subjects  who  were  map-only  in  the  low-fidelity  map  experiment.  RW  represents  the  real- 
world  subjects  fi’om  the  Goerger  experiment.  VE  represents  the  Goerger  study  VE 
participants.  VEl  is  those  subjects  who  were  in  the  VE  group  for  the  train-to-standard 
experiment.  VE2  is  the  VE  group  for  the  low-fidelity  map  experiment 

3.  Normalization  of  Data 

Some  of  the  participants  in  the  experiment  were  unable  to  complete  all  of  the  tasks 
because  of  the  one-hour  time  limit  during  execution  of  the  actual  course.  This  is  because 
some  of  the  measurements  occur  at  various  locations  spaced  out  over  time  and  space.  To 
make  the  subject’s  data  comparable,  many  of  the  measures  were  normalized  over  the 
number  of  controls  attempted  to  put  the  data  into  a  format  for  comparison  analysis.  This 
normalization  technique  was  used  in  evaluating  average  number  of  errors,  average  error 
distance,  average  distance  per  error,  and  average  mapcheck  scores  respectively. 

4.  Landmark  Knowledge 

A  subject’s  landmark  knowledge  was  determined  by  their  ability  to  locate  the 
control  points  during  the  experiment.  Subjects  were  given  one  point  for  each  control  they 
found  (points  had  to  be  found  in  order).  If  the  subject  was  timed  out  prior  to  completing 
the  course,  they  could  receive  partial  credit  for  the  next  point  on  their  route.  Subjects  who 
had  been  heading  to  the  next  point  and  who  were  on  their  planned  route  received  2/3- 
point  credit.  Subjects  who  were  off  their  planned  route  received  1/3  point  credit.  Subjects 
received  no  credit  for  points  that  they  did  not  visit  nor  attempt.  This  scoring  criterion  is 
the  same  as  in  the  original  Goerger  experiment  [GOER  98b].  This  score  was  combined 
into  each  subject’s  landmark  score. 

The  landmark  scores  for  all  seven  training  groups  are  shown  in  Figure  4.1. 
Statistically  the  means  are  not  different  to  our  a  standard  of  0.05,  F(6,35)  =  2.395,  P  = 
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0.054.  However,  statistical  differences  were  noted  between  the  VE  and  MAPI,  VE  and 
MAP2,  and  VE  and  VEl.  Observation  of  the  results  suggests  that  the  train-to-standard 
groups  perform  the  best  with  the  VEl  train-to-stahdard  group  achieving  a  perfect  result 
(all  subjects  finished  the  course  finding  all  the  controls).  The  map-only  train-to-standard 
was  not  far  behind  with  all  subjects  finding  eight  or  more  controls.  This  suggests  that 
subjects  who  trained  to  a  standard,  and  not  just  for  an  arbitrary  time,  learned  the  course 
better.  Furthermore,  those  that  had  time  to  explore  in  the  VE  were  able  to  gain  the  most 
landmark  knowledge.  It  also  lends  credibility  to  the  first  hypothesis  of  this  thesis. 
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Figure  4.1.  Box  Plot  of  Landmark  Score  (Group) 


The  strong  landmark  performance  of  the  MAP2  group  (map-only  group  low- 
fidelity  map)  does  not  fit  the  experiment’s  hypothesis.  The  most  likely  explanation 
involves  route  simplification.  This  explanation  is  described  in  Section  B. 

When  evaluating  the  results  strictly  from  the  macro-group  perspective,  the  analysis 
is  far  less  distinct,  with  F(2,35)  =  1.236,  P=  0.304.  We  are  thus  not  able  to  state  any 
conclusive  results  from  any  given  treatment  source  (RW,  VE,  or  MAP).  This  lack  of 
significance  is  not  surprising  given  the  very  strong  differences  noted  above  between  the 
various  treatment  groups.  Figure  4.2  shows  the  box  plot  of  the  macro-group. 
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Figure  4.2.  Box  Plot  of  Landmark  Score  (Rilacro-group) 

5.  Route  Knowledge 

For  this  thesis,  a  subject’s  route  knowledge  was  determined  by  evaluating  how 
well  they  foDowed  the  path  that  they  planned  during  their  initial  training  period  (see 
Chapter  in.  Section  A.  Training  Overview).  Three  different  measures  were  judged  to 
determine  exactly  how  well  the  subjects  kept  to  their  route. 
a.  Normalized  Error  per  Control 

First,  the  number  of  errors  committed,  averaged  per  control  point  (and 
normalized  as  mentioned  earlier)  was  conq)ared.  The  types  of  errors  were  discussed  in 
Chapter  III  Errors.  For  this  analysis,  only  mirror,  parallel,  and  compound  errors  were 
included.  Reorientation  errors  and  out-of-bounds  errors  were  included  in  the  map  check 
results,  to  be  discussed  following.  Individuals  who  made  fewer  errors  per  control  point 
were  considered  to  have  performed  better. 

Figure  4.3  shows  the  normalized  average  error  score  for  all  seven  of  the 
subject  groups.  All  three  of  the  map  groups  are  about  the  s^e  with  between  0.6  and  0.7 
errors  per  control  attempted.  This  is  followed  by  the  real  world  at  0.78.  The  biggest 
difference  between  the  groups  comes  from  the  VE  groups.  The  train-to-standard  VE 
outperformed  everyone  with  0.40  errors  per  control.  This  was  followed  closely  by  the 
low-fidelity  map  VE  group  at  0.48  and  the  original  VE  group  trailed  everyone  with  1.0 
errors  per  CP  atten^t. 
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Figure  4.3.  Box  Plot  for  Errors  per  Control  Attempted  (Group) 

The  means  between  groups  are  statistically  different,  F(6,35)  =  2.497,  P  =  0.0462, 
with  the  significant  differences  coming  between  the  VE  and  VEl  group,  VE  and  VE2, 
RW  and  VEl,  VE  and  MAPI,  and  VE  and  MAP2.  The  analysis  of  the  data  seems  to  show 
a  remarkable  improvement  for  the  VE  subjects  when  trained  to  a  standard  as  opposed  to 
just  a  timft  limit,  as  in  Goerger’s  experiment.  Direct  observation  of  the  results  suggests 
that  the  map-only  subjects  for  the  train-to-standard  condition  also  showed  improvement 
over  the  other  map-only  counterparts,  but  to  a  much  lesser  degree.  Subjects  who  train-to- 
standard  with  maps  had  slightly  lower  average  number  of  errors  compared  to  the  other 
map-only  subjects  (0.600  vs.  0.676[MAP]  vs.  0.644[MAP2]).  This  result  would  tend  to 
support  Goerger’s  theory  that  the  information  that  can  be  gained  from  a  map  will  level  off 
over  time  [GOER  98b].  These  results  seem  to  support  the  first  hypothesis  of  this  thesis; 
subjects  who  trained  to  standard  with  the  VE  not  only  found  more  controls  than  the  other 
groups  (see  Section  A.4.  Landmark  Knowledge)  but  made  fewer  errors  than  those  who 
trained  to  time.  This  superior  result  would  have  been  even  lower  had  not  one  VEl  subject 
(VEl -3)  twice  decided  to  go  down  follow  a  path  he  believed  to  be  wrong,  just  to  confirm 
that  it  was  wrong!  The  results  of  the  VE2  and  MAP2  groups  are  surprising  and 
unexpected.  A  possible  explanation  is  found  later  in  Section  B.  1 . 
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When  the  results  are  examined  using  the  macro-groups  (MAP,  VE,  and 
RW),  no  statistical  difference  between  groups  is  found,  F(2,35)  =  0.411,  P  =  0.6662. 
However,  direct  observation  shows  the  MAP  macro-group  with  a  much  smaller  standard 
deviation  over  the  course  of  the  experiments.  This  inches  that  regardless  of  how  long  they 
studied,  or  the  level  of  information  on  the  map,  the  map-only  group  performed  about  the 
same,  at  least  as  far  as  number  of  errors.  One  of  the  reasons  for  this  is  the  familiarity  of  the 
military  officers  in  this  study  with  maps  and  map  reading.  Figure  4.4  shows  the  box  plot  of 
the  macro-groups. 
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Figure  4.4.  Box  Plot  of  Errors  per  CP  Attempt  (Macro-Group) 

b.  Distance  Traveled  per  Error 

The  number  of  errors  a  subject  makes  is  just  one  factor  of  route 
knowledge.  Just  as  inportant  is  determining  how  long  it  takes  a  subject  to  determine  they 
have  made  an  error  and  recover.  While  no  exact  time  meeisurements  were  taken  to 
determine  how  long  this  took,  this  was  evaluated  in  terms  of  distance  traveled  firom  the 
point  of  the  error  until  the  individual  returned  to  their  planned  route  or  found  the  correct 
control.  It  was  possible  for  subjects  to  never  recover  fi:om  an  error;  either  because  time, 
ran  out,  or  because  15  minutes  elapsed  and  the  monitor  reoriented  them.  For  purposes  of 
this  test,  all  of  a  subject’s  error  distance  for  a  point  was  added  and  then  divided  by  the 
total  number  of  errors  for  that  point.  Adding  the  subject’s  totals  for  each  control  point  and 
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then  dividing  by  the  number  of  controls  attempted  (to  normalize  the  data)  gives  the 
average  distance  per  error  per  CP  attempted.  Goerger  called  this  the  Total  Error  Score 
[GOER  98b].  Figure  4.5  shows  the  results  of  this  split  by  training  group. 
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Figure  4.5.  Box  Plot  for  Total  Error  Score  per  CP  Attempt  (Group) 


The  results  of  the  analysis  of  means  is  not  statistically  significant  overall, 
F(6,35)  =  2.188,  P=  0.0743,  but  there  were  major  differences  between  the  VEl  and  VE, 
VE2  and  VE,  MAPI  and  VE,  MAP2  and  VE,  and  VEl  and  RW.  Direct  observation  of  the 
data  shows  that  subjects  of  the  train-to-standard  groups  (VEl,  MAPI)  not  only  made 
fewer  errors  than  other  groups  (as  mention  in  the  previous  section),  but  also  traveled  less 
distance  on  those  errors  before  they  corrected  themselves.  This  seems  to  show  that  these 
subjects  had  a  better  understanding  of  the  route  they  planned  to  follow  and  where  they 
were  on  that  route  at  all  times.  It  also  means  they  recovered  quickly  if  they  did  veer  off 
course.  The  low-fidelity  map  groups  also  performed  very  well.  The  rationale  for  their 
improvement  is  explained  in  more  detail  in  Section  B.  1 . 

The  results  of  the  macro-groupings  were  also  analyzed  to  see  if  any 
significance  could  be  found  from  the  overall  training  group.  With  a  variance  of  means  of 
F(2,35)  =  0.422,  P  =  0.662,  no  significant  effect  can  be  concluded  based  on  the  macro 
grouping.  Figure  4.6  shows  the  box  plot  of  the  distance  per  error  based  on  the  macro¬ 
groups.  While  no  statistical  conclusion  can  be  reached,  direct  observation  shows  that  the 
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map  as  a  whole  resulted  in  much  less  standard  deviation  among  the  various  groups.  Again, 
this  can  be  attributed  to  the  familiarity  of  the  subjects  with  map  reading. 
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Figure  4.6  Box  Plot  for  Total  Error  Score  per  CP  Attempt  (Macro-Group) 
c.  Map  and  Compass  Checks 

Subjects  had  their  map  check  scores  calculated  for  each  leg  of  the  course. 
As  mentioned  in  Chapter  HI  Map  Checks,  subjects  were  allowed  to  make  three  different 
kinds  of  map  checks  during  the  course  of  the  experiment.  These  were  map  only,  compass 
only,  or  map  and  compass  together.  These  checks  were  timed  and  recorded  by  the 
monitor.  Subjects  were  not  limited  in  the  number  of  map  checks  they  were  allowed  to 
take,  and  they  could  take  back-to-back  checks  if  they  desired  additional  time.  Subject’s 
instructions  were  to  “minimize”  the  number  of  map-checks,  which  clearly  meant  different 
things  to  different  people.  Some  subjects  refused  to  request  the  map,  even  if  they  realized 
they  had  left  their  route  or  when  they  were  confused.  Others  requested  the  map  at  each 
control,  even  if  they  had  been  performing  well. 

The  weights  of  the  mapcheck  scores  are  the  same  as  in  the  Goerger 
experiment  to  allow  for  comparison  of  results  [GOER  98b].  Subjects  who  took  a  map  or  a 
compass  check  got  one  point.  Map  and  compass  checks  combined  were  1.5  points,  since 
the  two  in  combination  allowed  more  information  to  be  gathered.  Subjects  who  desired  to 
change  their  route  were  assessed  a  0.5  point  penalty.  While  additional  information  can  be 
gained  from  the  map  during  the  one  minute  allotted  to  mark  the  new  route,  this  is  offset  by 
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the  relatively  limited  study  time  the  individual  has  to  plan  a  new  route.  Goerger  stated  that 
the  subject  must  already  have  knowledge  of  their  location  and  destination  in  order  to 
correctly  plan  an  effective  new  route  [GOER  98b].  Accordingly,  this  value  was  set  lower 
so  as  not  to  penalize  the  subjects  for  information  they  ideally  already  had.  The  marking  of 
a  new  route  from  a  misidentified  location  immediately  gave  the  subject  a  parallel  error 
when  they  started  to  move.  If  a  subject  left  the  course  boundaries,  they  were  given  a  two- 
point  error  because  they  demonstrated  a  lack  of  knowledge  of  that  area.  Knowing  they 
moved  out-of-bounds,  they  discover  that  their  current  mental  map  is  in  error.  This  lets 
them  know  that  they  are  not  where  they  think  they  are,  thus  giving  information  that  may 
not  be  gleaned  from  looking  at  the  map  alone.  Finally,  if  subjects  had  been  lost  for  15 
continuous  minutes  they  were  awarded  a  reorientation  error  with  3.0  points.  This  extreme 
penalty  is  because  they  were  shown  their  exact  location  on  the  map,  and  were  then  given 
the  opportunity  to  plan  a  new  route  to  the  next  control. 

The  lower  the  mapcheck  score,  the  better  the  overall  performance.  Figure 
4.7  shows  the  box  plot  of  the  groups  using  a  normalized  map  check.  This  number  was  the 
total  number  of  mapcheck  points  the  subject  was  awarded  divided  by  the  number  of 
controls  attempted.  The  means  between  training  groups  are  statistically  different  with 
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Figure  4.7.  Box  Plot  for  Normalized  Map  Check  Score  per  CP  Attempt  (Group) 
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F(6,35)  =  3.078,  P  =  0.0193.  There  were  significant  differences  between  the  means  of  the 
VEl,  VE2,  MAPI,  and  the  MAP2  groups  and  the  RW  and  VE  groups.  These  results 
indicate  two  different  things.  First,  in  the  case  of  the  low-fidelity  map  groups,  subjects 
tended  to  not  consult  the  map  as  much  because  of  the  scarcity  of  detail  on  the  map.  Also, 
those  subjects  tended  to  choose  simpler  routes  that  kept  to  the  exterior  boundary  roads 
and  thus  had  lesser  chance  to  make  parallel  errors  and  need  to  make  map  consultations. 
This  finding  is  explored  in  more  detail  in  section  B.  Second,  as  noted  in  the  original 
Goerger  experiment,  subjects  who  had  more  confidence  in  their  mental  maps  required  less 
checks  to  resolve  differences.  The  map-only  group  is  merely  demonstrating  the  same 
effect  as  shown  in  the  Goerger  experiment  [GOER  98b].  The  results  suggest  that  the 
train-to-standard  VE  group  was  also  able  to  develop  this  superior  mental  map  during 
training.  This  suggests  that  training-to-standard  developed  an  individual’s  mental  map 
more  than  the  hurried  one  hour  study  period  previously  allowed. 

6.  Survey  Knowledge 

As  explained  in  Chapter  n  Spatial  Knowledge,  survey  knowledge  is  considered  to 
be  the  highest  level  of  spatial  knowledge  and  is  the  most  difficult  to  acquire.  In  order  to 
determine  the  subject’s  level  of  survey  knowledge  of  the  environment,  three  different  tests 
were  conducted. 

a.  Wheel  Test 

Chapter  III,  Section  j.  Wheel  Test  describes  the  conduct  and  reasoning 
behind  the  wheel  test,  which  was  conducted  at  controls  two  and  four.  All  subjects  in  the 
experiments  conducted  for  this  thesis  performed  these  tests.  In  the  original  Goerger 
experiment,  four  subjects  failed  to  reach  control  point  four  and  thus  did  not  take  the 
second  wheel  test.  For  the  test,  the  subject’s  answers  were  compared  to  the  actual  angle 
of  the  controls.  The  absolute  value  of  the  difference  was  calculated  and  summed  for  each 
test  (CP2  and  CP4  respectively).  A  lower  angle  difference  equates  to  better  performance. 
Figure  4.8  shows  the  total  average  angular  difference  based  on  the  groups.  Figure  4.9 
shows  the  group  results  for  CP2  and  Figure  4.10  shows  the  CP4  results. 
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Figure  4.8.  Box  Plot  for  Average  Wheel  Test  Angular  Difference  (Group) 
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Figure  4.9.  Box  Plot  for  CP2  Wheel  Test  Angular  Difference  (Group) 
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Figure  4. 10.  Box  Plot  for  CP4  Wheel  Test  Angular  Difference  (Group) 


For  the  total  average,  the  means  between  the  groups  show  no  statistical 
difference,  F(2,35)  =  0.425,  P  =  0.859.  The  same  is  true  for  the  individual  tests  evaluated 
separately:  CP2  (F(2,35)  =  1.162,  P  =  0.354)  and  CP4  (F(2,31)=  2.488,  P  =  0.0514). 
Direct  observation  of  the  results  suggests  that  overall  there  was  no  discemable  pattern.  At 
CP4,  both  the  MAP2  and  VE2  averaged  the  least  angular  difference  of  any  of  the  groups. 
Perhaps  the  clear  depiction  of  the  depression  resulted  in  a  clearer  mental  map  at  this  point. 

As  a  further  check  regarding  survey  knowledge,  the  data  was  examined  at 
the  macro-group  level.  Figure  4.11  shows  the  box  plot  of  the  results.  All  of  the  groups 
averaged  a  total  of  about  27  degrees  off  in  their  estimation.  While  not  statistically  valid 
(F(2,35)  =  0.231,  P  =0.794  ),  direct  observation  suggests  that  subjects  who  used  only  the 
map  had  less  deviation  in  their  answers,  resulting  in  more  uniform  performance.  This  does 
not  show  that  any  group  had  better  survey  knowledge. 
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Figure  4. 1 1.  Box  Plot  for  Average  Wheel  Test  Angular  Difference  (Macro-Group) 
b.  White  Board  Test 

Upon  the  conclusion  of  the  experiment,  every  subject  was  given  the 
whiteboard  test  (see  Chapter  III  White  Board  Test).  The  analysis  of  the  whiteboard  results 
allowed  a  subject’s  level  of  exocentric  survey  knowledge  to  be  evaluated.  This  was  done 
by  following  the  same  procedures  in  the  Goerger  experiment  [GOER  98b].  The  angle 
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between  the  control  points  was  measured  by  determining  the  distance  the  subject  judged 
the  points  to  be  apart  in  the  XY  plane.  This  distance  was  normalized  using  the  total 
distance  and  then,  using  the  Pythagorean  Theorem,  the  angles  between  successive  points 
were  calculated.  The  subject  angle  results  were  compared  to  the  actual  course  angles. 
Figure  4. 12  shows  a  box  plot  of  the  resulting  angular  differences  by  group. 
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Figure  4.12.  Box  Plot  for  Average  Whiteboard  Angular  Difference  (Group) 


Statistically  there  was  no  difference  between  the  means  with  F(6,35)  = 
0.789,  P  =  0.586.  Direct  observation  seems  to  show  that  the  real  world  had  less  variance 
between  their  answers,  but  by  no  means  did  they  perform  better  than  any  other  group. 
Figure  4. 13  analyzes  the  results  based  on  the  macro-group. 
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Figure  4.13.  Box  Plot  for  Average  Whiteboard  Angular  Difference  (Macro-Group) 
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No  statistical  variation  is  noticeable  (F(2,35)  =  0.480,  P  =  0.624),  nor  do 
any  differences  appear  in  direct  observation.  The  results  seem  to  suggest  that  there  is  no 
difference  between  the  groups  in  survey  knowledge  based  on  the  angle  error. 

The  distance  difference  from  the  normalized  distance  of  a  leg  and  the  user’s 
normalized  distance  from  the  whiteboard  was  also  analyzed  and  compared.  As  with  the 
angle  analysis,  no  statistical  differences  were  noted  among  either  the  groups  (F(2,6)  = 
2.19,  P  =  0.0732)  or  the  macro-groups  (F(2,35)  =  1.37,  P  =  0.268).  Figure  4.14  shows  the 
box  plot  for  the  group  average  distance  difference.  Direct  observation  of  the  data  shows 
that  the  MAP2  group  averaged  lower  errors  on  this  task.  The  VEl  group  also  had  the 
most  variation,  suggesting  that  the  VE  did  not  play  a  significant  role  in  imparting  survey 
knowledge.  It  also  suggests  that  the  lack  of  means  of  distance  estimation  in  the  VE 
particularly  hurt  those  who  spent  extended  time  in  the  model. 
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Figure  4.14.  Box  Plot  for  Average  Whiteboard  Distance  Difference  (Group) 


Overall,  there  does  not  seem  to  be  a  large  difference  between  the  various 
groups  in  the  survey  knowledge  evaluated  by  the  whiteboard.  This  may  be  in  part  because 
this  test  was  given  at  the  very  end  of  the  course,  after  all  individuals  had  been  exposed  to 
the  area.  This  exposure  may  have  equalized  any  benefit  or  hindrance  imparted  by  the 
various  training  methods.  It  might  be  better  for  future  experiments  to  conduct  the 
whiteboard  test  prior  to  running  the  course,  so  that  the  subject  will  not  have  modified  their 
mental  map  based  on  the  actual  terrain. 
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c.  Unplanned  Route  Execution 

If,  and  only  if,  the  subjects  successfully  complete  the  course  in  one  hour 
were  they  given  the  unplanned  route  test.  For  this  task  the  subject  would  have  to  first 
explain  how  they  would  go  firom  their  current  location,  control  point  nine,  to  control  point 
four.  They  had  to  give  this  explanation  without  first  looking  at  a  map.  The  subjects  were 
then  given  ten  minutes  to  move  from  CP9  to  CP4.  Most  of  the  subjects  planned  this  route 
to  allow  them  to  go  over  terrain  they  had  been  over  once  before,  whether  from  a 
previously  planned  route,  or  firom  travelling  through  the  area  while  in  error.  For  the 
experiments  done  for  this  thesis,  every  subject  who  reached  control  point  nine  was  able  to 
successfully  con^lete  the  unplanned  route  in  the  time  allotted  and  none  made  errors  off 
their  planned  route.  This  was  also  the  result  of  the  Goerger  study  [GOER  98b].  While 
there  was  a  slight  difference  in  the  time  subjects  took  to  accomplish  the  task,  it  was  not 
significant. 

These  results  seem  to  indicate  there  is  no  discemable  difference  in 
acquiring  survey  knowledge  based  on  training  method.  However,  because  the  test  was  not 
given  to  every  subject,  it  is  impossible  to  draw  any  conclusions  concerning  this  task 
performance  and  training  grouping.  There  is  also  the  possibility  that  since  subjects  had 
actually  traveled  the  ground,  that  any  survey  knowledge  gained  was  the  result  of  this 
direct  exposure. 

One  interesting  caveat  to  this  experiment:  One  test  subject  (Ml-1)  was 
timed  out  51  meters  short  of  CP9  and  was  on  his  route  when  time  ended.  Because  he  was 
so  close  to  finishing,  the  subject  was  given  the  unplanned  route  test  (though  because  he 
did  not  finish,  hds  results  were  not  included).  Surprisingly,  the  subject  was  not  able  to 
accon^lish  the  task.  Instead,  he  become  disoriented  and  missed  his  planned  turn.  Whether 
this  result  was  due  to  an  incomplete  mental  map  or  some  other  factor  is  unknown. 

7.  Navigational  Peifonnahce  by  Training  Condition 
The  results  of  the  analysis  do  indicate  statistical  significance  based  upon  the 
training  condition.  The  results  show  that  those  who  trained  using  the  VEl,  VE2,  MAPI, 
and  MAP2  generally  performed  landmark  and  route  tasks  better  than  the  original  VE  and 
RW  group.  Direct  observation  of  the  results  showed  the  train-to-standard  groups 
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performed  better  than  the  Goerger  training  groups  (RW,  MAP,  and  VE)  and  the  low 
fidelity  map  group  (VE2,  MAP2).  Direct  observation  also  showed  that  the  train-to- 
standard  VE  group  appeared  to  outperform  all  other  groups  in  the  route  and  landmark 
tasks. 

Caution  is  advised  when  using  these  for  conclusion  for  a  number  of  reasons.  The 
relatively  small  sample  size  in  each  group  allows  the  results  to  be  thrown  off  by  one  or 
two  outlying  results.  Also,  it  is  difficult  to  compare  the  results  of  broad  categories  of 
errors  and  distances  given  the  vastly  different  routes  subjects  used.  If  subject  A  made  his 
errors  on  a  short  but  difficult  route,  how  comparable  is  the  mistake  to  subject  B’s  mistake 
on  a  longer  easier  route?  Another  reason  for  caution  is  that  the  standards  for  judging 
errors  in  the  study  required  navigation  to  standards  that  are  vastly  different  from  most 
mihtary  subject’s  experience.  Most  military  navigators  use  terrain  association  and  do  not 
hesitate  to  perform  quick  detours  to  confirm  they  are  on  their  route.  In  this  experiment, 
this  technique  results  in  an  error.  Many  subjects  would  also  adopt  a  wide,  circular  search 
pattern  for  the  controls,  adding  error  distance. 

The  low  fidelity  map  group’s  (MAP2,  VE2)  switching  to  the  boundary  road 
approach  was  unforeseen,  and  resulted  in  their  better  performance  than  the  original 
Goerger  groups.  While  travelling  the  boundary  road  was  not  always  considered  a  simpler 
route  (see  Appendix  L)  it  was  easier  to  follow  than  the  numerous  twisting  interior  trails. 
Perhaps  prohibiting  use  of  the  boundary  roads  would  have  produced  results  more  in 
keeping  with  the  second  hypothesis  of  this  thesis.  However,  because  no  such  restriction 
existed  for  the  other  groups  this  would  have  made  the  results  mcomparable. 

With  regards  to  survey  knowledge  there  was  no  discemable  difference  between 
either  the  training  groups  or  the  macro  groups.  This  may  be  in  part  due  to  the  nature  in 
which  most  of  the  tests  were  conducted.  Since  only  those  who  finished  the  course  were 
given  the  unplanned  route  test,  only  those  with  a  better  overall  picture  of  the  environment 
were  able  to  attenpt  this  test.  It  is  no  surprise  that  they  completed  it.  The  exposure  to  the 
actual  course  also  may  have  blurred  potential  distinctions  between  the  training  groups  in 
the  whiteboard  results.  Future  research  may  need  to  focus  more  on  the  survey  task  directly 
in  order  to  better  discern  the  differences,  if  any. 
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B.  DISCUSSION 

1.  Effectof  Lowering  Map  Fidelity. 

As  explained  in  Chapter  III,  Section  N.  Training,  the  second  experiment  was 
conducted  to  examine  possible  inherent  bias  in  the  highly  detailed  map  given  subjects  of 
the  earlier  experiments.  The  expected  results  were  that  everyone  would  perform  more 
poorly,  but  that  the  VE  group’s  performance  drop  would  be  less  severe  than  that  of  the 
map-only  group.  As  was  noted  earlier  in  Sections  A.4,  A.5,  and  A.6,  the  low  fidelity  map- 
only  and  VE  groups  did  better  than  their  corresponding  groups  in  the  original  Goerger 
experiment.  What  would  explain  this? 

First,  it  should  be  noted  that  while  the  map  had  a  lower  overall  fidelity,  one  thing 
the  map  improved  upon  was  in  clearly  showing  control  point  four  was  in  a  depression. 
Nevertheless,  how  much  effect  did  this  clarification  have  on  performance?  Figure  4.15 
shows  the  seven  groups  number  of  errors  on  control  point  four. 
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Figure  4. 1 5.  Box  Plot  for  Number  of  Errors  CP4  (Group) 

This  figure  shows  that  on  average,  the  low  fidelity  map  groups  made  one  less  error 
than  the  original  VE  group.  In  fact,  the  original  VE  group’s  average  for  CP4  is  twice  that 
of  their  overall  average  (see  Figure  4.3).  This  is  a  higher  percentage  than  for  the  other 
groups.  Both  the  MAPI  train-to-standard  and  the  MAP  group  also  had  much  higher  error 
rates  for  this  control  than  their  other  points.  Note  that  neither  the  low  fidelity  map,  nor  the 
extended  exposure  to  the  virtual  environment  for  VEl,  eliminated  errors.  The  real  effect  is 
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seen  in  Figure  4.16,  which  shows  the  average  amount  of  error  distance  for  each  group  at 
CP4. 
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Figure  4. 16.  Box  Plot  for  Average  Distance  per  Error  CP4  (Group) 


From  these  results,  we  clearly  see  that  the  original  VE,  MAP,  and  RW  had  much 
higher  error  distances  at  CP4  compared  to  the  low  fidelity  map.  This  raised  their  overall 
total  error  score  considerably.  The  train-to-standard  VE  group  escaped  this  problem,  most 
likely  due  to  the  extended  exposure  to  the  area  during  training  allowing  them  to  overcome 
the  poor  map  representation. 
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Figure  4.17  shows  the  map  check  scores  for  CP4.  From  the  box  plot,  the  results 
show  that  every  group  other  than  the  low  fidelity  map  groups  and  the  train-to-standard 
VEl  group  had  much  higher  map  checks  score  than  average  at  CP4.  Since  the  low  fidelity 
group  did  not  have  this  large  statistical  jump,  their  overall  performance  was  lower. 

The  other  reason  for  the  in^roved  performance  is  that  the  low  fidelity  map  groups 
adjusted  their  routes  based  on  the  information  provided.  Since  this  group  was  provided 
almost  no  information  about  the  interior  trails  in  the  area,  they  tended  to  avoid  internal 
travel  whenever  possible  and  stuck  to  the  boundary  roads  more  than  the  other  groups, 
even  for  short  movements.  While  staying  on  the  boundary  roads  was  not  always 
considered  the  easier  route  (see  Appendix  L)  it  did  have  the  effect  of  reducing  complexity 
and  reducing  the  chance  of  parallel  errors.  There  tended  to  be  limited  trail  and  path 
intersections  with  the  boundary  road,  so  individuals  could  easily  discern  their  location. 
Since  those  intersections  that  did  exist  were  prominent,  the  low  fidelity  group  tended  to 
have  good  attack  points  when  they  left  the  boundary  road  to  begin  their  search.  Figures 
4.18  and  4.19  show  the  box  plots  of  the  planned  difficulty  of  the  routes  for  the  two 
groups.  While  there  is  no  statistical  difference,  direct  observation  suggests  that  the  low 
fidelity  groups  plaimed  easier  routes.  Again,  remember  that  using  the  boundary  road  was 
not  always  considered  the  easier  route  since  in  many  cases  it  added  significant  distance. 
Section  B.2.  Route  Complexity  shows  how  simpler  routes  improve  performance. 
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Figure  4. 18.  Box  Plot  for  Lisp  Planned  Route  Difficulty  (Map  Fidehty) 
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Figure  4. 19.  Box  Plot  for  Banker  Planned  Route  Difficulty  (Map  Fidelity) 

Many  subjects  were  aware  that  their  strategy  of  using  the  boundary  roads  was 
adding  distance.  Within  VE2,  both  subjects  who  failed  to  find  CP7  determined  that  they 
were  “eating”  too  much  time  by  using  the  boundary  roads.  They  planned  to  incorporate  a 
shortcut  while  enroute  fi:om  CP6  to  CP7  along  the  boundary  road  by  cutting  through  the 
woods  to  save  time.  In  both  cases,  the  subjects  never  recovered  fi'om  their  “shortcut.” 

Figure  4.20  clearly  shows  that  the  low  fidelity  map  group  conducted  less  map 
checks  as  well.  This  was  most  likely  due  to  the  lack  of  detail  on  the  map. 
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Overall,  the  fidelity  of  the  map  had  no  effect  on  survey  knowledge  tasks.  Figure 
4.21  shows  the  box  plots  of  the  average  angular  difference  from  the  wheel  test  (discussed 
in  Section  A. 6  previously).  No  difference  is  discemable.  Likewise,  there  is  no  difference 
for  CP2.  This  is  shown  in  Figure  4.22. 
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Figure  4.21.  Wheel  Test  Angular  Difference  (Map  Fidelity) 
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Figure  4.22.  Average  Wheel  Test  Angular  Difference  CP2  (Map  Fidelity) 


There  is  a  statistical  difference  between  the  means  at  CP4,  however,  F(l,31)  = 
4.736,  P  =  0.0373.  Observation  of  the  data  suggests  that  the  1:24,000  scale  map  users  had 
less  angular  difference  than  those  with  the  1:5,000  scale  map.  Figure  4.23  shows  this 
result.  The  low  fidelity  scale  map  users  averaged  ten  degrees  less  difference  than  the  other 
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group.  The  likely  explanation  for  this  was  the  confusion  that  many  1:5,000  scale  users 
suffered  over  the  location  of  CP4.  In  mistakenly  looking  for  CP4  on  a  hilltop,  when  they 
finally  located  CP4,  they  had  very  recently  changed  their  mental  map.  This  may  have 
resulted  in  their  overall  worse  performance. 
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Figure  4.23.  Average  Wheel  Test  Angular  Difference  CP4  (Map  Fidelity) 


Examining  the  results  of  the  whiteboard  test,  we  see  no  statistical  difference 
between  those  with  the  high  fidelity  map  and  the  low  fidelity  map.  Direct  observation 
suggests  that  the  low-fidelity  map  averaged  slightly  less  error. 
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Figure  4.24.  Average  Whiteboard  Angular  Difference  (Map  Fidelity) 
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2.  Route  Complexity 

As  mentioned  in  the  preceding  section,  the  fact  that  the  low  fidelity  map  groups 
shifted  to  the  boundary  roads  resulted  in  giving  them  better  performance  than  the  original 
Goerger  groups.  In  many  cases,  following  the  boundary  roads  was  not  considered  the 
“sinplest”  route.  What  correlation,  if  any,  exists  between  route  difficulty  and 
performance? 

Figure  4.25  shows  the  regression  plot  of  the  planned  route  difficulty  of  both  the 
Lisp  and  Banker  scales  versus  the  Landmark  results  (see  Appendix  L).  Direct  observation 
of  the  plots  suggests  that  as  the  planned  difficulty  of  the  route  went  up,  the  subject’s  score 


Y  =  11.186  -  2.024  *X;R'2=. 191  Y  =  10.631  -  1.685 'X;  1^2  =  .137 

Figure  4.25.  Regression  Plot  for  Landmark  Results  vs.  Planned  Route  Difficulty 

went  down.  These  results  are  statistically  significant  with  the  lisp  results  F=  7.78,  P  = 
0.0086,  and  Banker  F=  5.223,  P  =  0.0289.  This  suggests  that  subjects  with  “easier”  routes 
were  less  likely  to  make  errors  they  could  not  recover  from,  nor  were  they  disoriented  as 
much.  It  also  suggests  that  “easier”  routes  were  more  readily  memorized  and  recalled. 
Similar  results  occurred  for  the  errors  per  control  point  attempted,  with  both  scales 
achieving  statistically  valid  results  (Banker:  F=  7.815,  P  =  0.0086,  Lisp:  F=  10.584,  P  = 
0.0026).  In  both  cases,  subjects  with  easier  routes  made  fewer  errors.  This  is  shown  in 
Figure  4.26,  with  the  lower  number  of  errors  indicating  better  performance. 
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Figure  4.26.  Regression  Plot  for  Errors  per  CP  Attempt  vs.  Route  Difficulty 


The  results  of  simple  regression  of  the  route  difficulty  versus  both  the  normalized 
map  check  score  and  the  total  error  score,  while  not  resulting  in  statistically  valid  results, 
did  suggest  a  similar  pattern.  Direct  observation  of  the  data  seemed  to  show  that  those 
with  easier  routes  made  less  map  checks  and  traveled  less  distance  per  error  than  those 
following  routes  that  were  more  difficult. 

These  results  indicate  that  the  difficulty  of  the  route  that  subjects  planned  did  have 
an  effect  on  their  route  and  landmark  task  performance.  They  also  provide  anecdotal 
support  for  the  first  hypothesis.  In  all  three  experiments  in  this  series,  real-world  and  VE 
subjects  with  a  one  hour  planning  limit  often  ran  out  of  time  during  their  training  before 
adequately  planning  their  route.  This  was  often  caused  by  the  subject  actively  exploring 
possible  routes  firom  the  start,  as  opposed  to  solely  using  the  map.  Because  they  spent 
much  of  their  time  exploring,  they  often  had  extremely  limited  time  to  plan  and  memorize 
the  final  legs  of  their  route.  Often  this  led  to  them  just  drawing  a  “straight-line”  between 
points,  as  opposed  to  finding  an  easier  route.  Such  “straight  line”  routes  were  almost 
always  among  the  most  difficult,  and,  firom  the  above  results,  the  most  error  prone.  Thus, 
the  one-hour  time  limit  for  the  VE/real-world  tended  to  force  harder  routes,  which  m  turn 
resulted  in  poorer  performance.  Removing  the  time  constraint  resulted  in  easier  routes  and 
better  performance. 
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The  user’s  route  difficulty  had  no  bearing  on  their  survey  task  performance. 
Regression  analysis  of  route  difficulty  versus  angular  difference  resulted  in  no  significant 
pattern.  Nor  were  there  valid  differences  in  the  whiteboard  test. 

3.  Effect  of  User  AbiKty  Level 

Individual  differences  can  have  a  large  effect  on  performance,  and  one  of  the  major 
differences  between  the  subjects  in  this  experiment  was  their  level  of  navigational 
experience.  Subjects  evaluated  their  skill  level  both  written  and  by  using  a  bar-line.  The 
bar-line  was  used  for  our  comparison  analysis,  as  no  subjects  evaluated  themselves  as 
beginners  (possibly  due  to  bias  against  the  word  “beginner”)  with  their  words,  but  several 
fell  into  this  category  by  the  bar-line. 

Examining  the  landmark  results  based  on  the  subjects,  there  is  no  statistically  valid 
difference  between  the  three  ability  levels,  F(2,35)=  1.938,  P  =  0.1605.  However,  direct 
observation  of  the  data,  shown  in  Figure  4.27,  suggests  that  those  subjects  in  the  expert 
category  found  more  control  points.  The  expert  group  also  had  less  variation.  Beginners 
averaged  the  least  number  of  controls  found,  and  the  most  variation. 

Box  Plot 

Split  By:  Seif  Eval  (Bar-Line) 
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Figure  4.27.  Landmark  Score  (Self-Eval) 

Figure  4.28  displays  both  the  errors  per  control  attempted  and  the  total  error  score 
for  the  user’s  navigational  ability  level.  The  lower  score  represents  better  performance. 
Neither  analysis  resulted  in  statistically  valid  difference  of  means.  Direct  observation  of  the 
results  suggests  that  expert  classification  subjects  traveled  less  distance  after  making  an 
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error  before  they  realized  their  mistake.  The  data  also  suggests  that  beginners  made  the 
most  errors  per  CP  attempted. 


350 

300 


Box  Plot 
Split 


Box  Plot 

Split  By:Self-Bral 


1.6  H 


Figure  4.28.  Total  Error  Score/Attempt  and  Errors  per  CP/Attempt  (Self-Eval) 


The  results  of  the  normalized  map  check  scores,  analyzed  by  self-evaluated  ability 
group,  are  shown  in  Figure  4.29.  There  is  no  correlation  apparent  upon  direct  observation. 
Intermediate  ability  users  had  the  widest  variation.  Beginners  averaged  the  same  map 
check  score  as  experts,  but  for  different  reasons.  Experts  were  less  likely  to  be  lost  and 
require  a  map  check.  Beginners  were  less  likely  to  realize  they  needed  one. 
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Figure  4.29.  Normalized  Map  Check  Score/Attempt  (Self-Eval) 
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No  differences  were  determined  in  the  survey  knowledge  of  subjects  based  on  their 
self-evaluated  ability  level.  Nor  was  there  a  statistical  difference  in  the  difficulty  level  of 
the  subjects  chosen  routes. 

4.  Spatial  Ability  Post-Hoc  Correlation 

Goerger  found  a  significant  difference  in  the  performance  between  the  high  spatial 
ability  subjects  and  the  low  spatial  ability  subjects  [GOER  98b].  These  results  were 
confirmed  by  this  experiment.  Examining  the  landmark  results  based  on  the  subject’s  GZ 
spatial  ability  showed  a  significant  difference  between  the  means  based  on  performance, 
F(l,35)  =  8.221,  P  =  0.0072.  These  results  are  shown  in  the  box  plot  in  Figure  4.30. 
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Figure  4.30.  Landmark  Score  (GZ  Ability  Group) 


Figure  4.31  shows  the  total  error  score  per  CP  attempted  for  the  GZ  ability  group. 
The  groups  show  statistical  significance,  F(l,35)  =  7.698,  P  =  0.0092,  Subjects  who 
scored  higher  on  the  GZ  traveled  less  distance  per  error,  meaning  they  realized  their  errors 
faster  than  low  GZ  subjects  and  recovered  sooner.  Direct  observation  of  high  GZ  ability 
subjects  suggests  they  made  less  overall  errors,  as  shown  in  Figure  4.32.  These  results 
were  not  significant  at  F(l,35)  =  3.393,  P  =  0.0557.  Overall,  the  results  suggest  that  high 
ability  subjects  made  fewer  errors,  and  recovered  faster  firom  their  errors,  than  low  GZ 
subjects.  This  in  turn  enabled  them  to  attempt  more  controls  and  achieve  higher  landmark 
scores.  The  exact  cause  of  their  better  performance,  whether  better  memory,  mental  map. 
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route  selection,  or  overall  organization,  is  not  precisely  determined  at  this  point.  Future 
research  should  examine  exactly  why  those  with  high  spatial  ability  perform  better. 
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Figure  4.31.  Total  Error  Score/Attempt  (GZ  Ability  Group) 
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Figure  4.32.  Errors  per  CP  Attempt  (GZ  Ability  Group) 


High  GZ  ability  subjects  also  had  lower  normalized  map  check  scores  than  their 
counterparts  by  a  statistically  significant  margin,  F(l,35)  =  4.728,  P  =  0.0465.  This  is 
shown  in  Figure  4.33.  These  results  suggest  that  high  GZ  score  subjects  realized  their 
errors  quickly  enough  that  they  did  not  need  a  map  to  confirm  this.  There  was  no 
difference  in  the  normalized  non-error  map  check  scores  between  the  groups  (F(l,35)  = 
0.712,  P  =  0.4048).  These  results  are  shown  in  Figure  4.34.  This  result  casts  doubt  on 
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Goerger’s  assertion  that  the  low  GZ  map  check  score  is  a  result  of  better  memory  or  more 
confidence  in  their  route.  High  GZ  subjects  averaged  roughly  the  same  number  of 
maintenance  checks  per  control  as  the  low  ability  group. 
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Figure  4.33.  Normalized  Map  Check  Score/Attempt  (GZ  Ability  Group) 
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Figure  4.34.  Normalized  Maintenance  Checks/Attenqpt  (GZ  Ability  Group) 


There  was  no  difference  in  the  map  reading  skills  of  the  two  groups,  however 
analysis  showed  that  the  high  GZ  group  did  select  less  difficult  routes.  This  is  shown  in 
Figure  4.35.  This  significant  difference  (Banker  F=  7.93,  P  =  0.0081,  Lisp  F  =  8.141,  P  = 
0.0074)  suggests  that  high  GZ  ability  subjects  were  able  to  recognize  easier  routes  and 
simpler  movements  than  their  counterparts.  They  were  able  to  discern  which  routes 
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offered  easUy  recognizable  features  and  landmarks  that  could  be  used  as  progress 
indicators.  This  allowed  them  to  confirm  their  position  as  they  went,  helping  to  reduce 
their  overall  number  of  errors. 
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Figure  4.35.  Banker  and  Lisp  Route  Difficulty  (GZ  Ability  Group) 


With  regards  to  survey  knowledge,  there  was  no  difference  between  the  GZ  ability 
levels  and  the  whiteboard  results.  There  was  no  statistical  difference  between  the  wheel 
test  angular  deviations  (F  (1,35)  =  1.364,  P  =  0.2513).  Direct  observation  of  the  wheel 
test  results  suggests  that  high  ability  subjects  did  average  slightly  less  error  than  their  low 
ability  counterparts.  This  is  shown  in  Figure  4.36.  This  implies  that  high  spatial  ability 
allowed  subjects  to  better  orient  their  mental  map  of  the  environment  on  demand  to 
determine  the  location  of  other  landmarks  in  regards  to  their  own  position. 
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Figure  4.36.  Wheel  Test  Angular  Difference  (GZ  Ability  Group) 
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5.  Analysis  Based  on  Santa  Barbara 

Goerger,  in  his  experiment,  did  not  determine  any  discemable  results  based  on  the 
Santa  Barbara  sense  of  direction  scale  [GOER  98b].  The  results  of  this  experiment  did 
show  a  significant  difference  between  the  results.  Note,  however,  the  very  low  number  of 
subjects  who  were  classified  as  low. 

Figure  4.37  displays  the  landmark  results  based  on  the  Santa  Barbara  scale.  The 
results  are  statistically  different  with  F(l,35)  =  7.093,  P  =  0.01 19. 
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Figure  4.37.  Landmark  Score  (Santa  Barbara  H/L) 

Figure  4.38  shows  the  box  blot  for  the  errors  per  CP  attempted  based  on  the 
subject’s  Santa  Barbara  results.  The  analysis  of  means  shows  a  significant  difference 
between  the  two  classifications,  F(l,35)  =  19.240,  P  =  0.0001.  Looking  at  the  results  of 
these  two  plots  shows  that  those  classified  as  high  ability  by  the  Santa  Barbara  averaged 
half  the  number  of  errors  of  the  low  ability  group.  Because  they  made  fewer  errors,  they 
appeared  less  inclined  to  lose  their  way,  and  in  turn,  this  helped  them  find  more  controls. 
Also  shown  in  Figure  4.38  is  the  total  error  score.  While  these  results  are  not  statistically 
different  (F  =  2.333,  P  =  0.264),  direct  observation  does  suggest  that  the  high  Santa 
Barbara  group  traveled  less  distance  per  error  before  recovering. 
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Figure  4.38.  Errors  per  CP  Attempt  and  Total  Error  Score  /Attenq)t  (Santa  Barbara) 


The  data  showed  no  statistically  significant  difference  between  the  Santa  Barbara 
levels  and  the  normalized  inap  check  scores  or  the  number  of  maintenance  checks  the 
subject  made.  However,  there  was  a  significant  difference  between  the  means  in  the  level 
of  route  difBculty  between  the  groups  (Banker  F  =  4.513,  P  =  0.0412,  Lisp  F  =  5.84,  P  = 
0.0243).  This  result  is  shown  in  Figure  4.39  with  the  lower  score  being  the  easier  route. 
Observation  of  these  results  indicate  that  low  Santa  Barbara  subjects  tended  to  choose 
more  difficult  routes.  This  choice  in  turn  may  have  caused  them  to  make  more  errors. 
These  subjects  may  have  lacked  intermittent  landmarks  to  help  them  confirm  they  were  on 
track,  allowing  them  to  travel  further  before  they  recovered  fi-om  errors. 
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Figure  4.39.  Banker  and  Lisp  Route  Difficulty  (Santa  Barbara) 
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There  were  no  differences  between  the  groups  for  the  wheel  test  survey  tasks. 
Neither  the  high  nor  the  low  group  performed  better  on  the  whiteboard.  Only  one  low 
ability  subject  completed  the  course,  but  that  subject  was  able  to  complete  the  unplanned 
route  task  to  standard  so  no  conclusions  can  be  drawn  there  as  well.  There  appears  to  be 
no  relation  between  a  subject’s  survey  knowledge  and  their  Santa  Barbara  rating. 

6.  Simulator  Sickness 

Goerger  had  commented  on  user’s  having  simulator  sickness  during  the  conduct  of 
the  experiments  [GOER  98b].  Of  the  ten  subjects  who  used  the  VE  in  this  experiment, 
only  one  complained  of  simulator  sickness.  That  individual  described  feeling  sick  but  said 
he  attributed  it  to  a  lack  of  food  prior  to  beginning  the  experiment.  The  subject  had  found 
he  had  to  take  frequent  breaks  until  it  became  quite  uncomfortable.  The  subject,  who  was 
part  of  the  train-to-standard  group,  was  given  20  minutes  to  go  eat.  Upon  returning,  all 
symptonos  of  simulator  sickness  disappeared  and  he  was  able  to  successfully  continue  the 
experiment  without  hindrance.  No  other  VE  subjects  experienced  any  synptoms,  though 
many  spent  considerable  time  training  (one  subject  trained  almost  three  hours).  Remember 
that  all  of  the  train-to-standard  subjects  had  to  con:q)lete  the  straight  run  through  of  the 
course  in  the  VE  as  part  of  the  pre-course  test  prior  to  being  allowed  on  the  course.  This 
would  tend  to  suggest  that  simulator  sickness  is  largely  due  to  individual  differences  and 
the  physical  condition  of  the  subjects  at  the  time  of  the  experiment. 

7.  Distinguishing  Benefits  Between  Groups. 

The  analysis  of  the  data  so  far  has  shown  that  there  are  both  differences  between 
the  spatial  knowledge  gained  based  on  training  group  and  based  on  spatial  ability.  The 
question  that  is  often  asked  is  whether  one  training  method  is  better  for  training  a 
particular  type  of  spatial  ability  individual?  In  other  words,  is  it  better  to  train  a  low  spatial 
ability  person  using  a  map  or  by  the  VE?  In  order  to  answer  this  question  the  results  of  the 
error  per  CP  attempt  and  total  error  score  were  examined  broken  down  by  group  and  GZ 
ability.  Table  1  shows  the  means  of  the  errors  per  CP  results.  No  significant  statistical 
results  can  be  concluded  from  the  data.  However  direct  observation  of  the  data  does 
suggest  that  there  was  a  smaller  difference  between  the  results  of  the  low  ability  and  high 
ability  subjects  who  used  the  VE  than  those  who  used  the  map  or  were  real  world 
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subjects.  Table  1  shows  that  the  low  spatial  ability  map-only  groups  averaged  0.3  errors 
more  per  CP  than  the  high  ability  GZ  subjects  in  the  same  training  group.  The  real  world 
had  an  even  larger  difference,  with  the  low  ability  averaging  0.6  errors  more  per  CP  than 
high  ability  subjects.  In  contrast,  the  low  GZ  VE  subjects  averaged  only  0.1  to  0.2  errors 
more  than  their  high  GZ  counterparts.  This  suggests  that  low  spatial  ability  subjects 
perform  closer  to  their  high  ability  counterpeirts  when  trained  using  the  VE  at  least  as  far 
as  the  number  of  errors  committed.  This  may  be  because  of  the  advantages  high  spatial 
ability  subjects  have  in  iiKntally  rotating  the  map  to  envision  what  the  terrain  will  look 
like,  which  low  GZ  subjects  may  lack.  In  the  VE,  subjects  did  not  have  to  do  this  mental 
translation  and  thus  all  performed  to  roughly  the  same  level. 


Descriptive  Statistics 
Split  By;  Group,  Ability  Group 


Mean 

Std.  Dev. 

Std.  Bror 

Count 

&rors  P&r  CP  Attempt,  Total 

.660 

.331 

.056 

35 

Brors  Per  CP  Attempt,  MAP,  Hgh 

.556 

0.000 

0.000 

3 

&rors  Rsr  CP  Attempt,  MAP,  Low 

.857 

.202 

.143 

2 

B'rors  Fter  CP  Attempt,  MAPI ,  High 

.  .778 

.157 

.111 

2 

Brors  Per  CP  Attempt,  MAPI,  Low 

.481 

.170 

.098 

3 

Brors  Pfer  CP  Attempt,  MAP2,  High 

.583 

.056 

.028 

4 

Brors  Fter  CP  Attempt,  MAR2,  Low 

.889 

• 

• 

1 

Brors  Per  CP  Attempt,  RW,  High 

.444 

.157 

.111 

2 

Brors  Per  CP  Attempt,  RW,  Low 

1.014 

.638 

.368 

3 

Brors  Fter  CP  Attempt,  VB  High 

.929 

.101 

.071 

2 

Brors  Fter  CP  Attempt,  VB  Low 

1.102 

.489 

.283 

3 

Brors  Fter  CP  Attempt,  VE1,  High 

.333 

.192 

.111 

3 

Brors  Fter  CP  Attempt,  VE1,  Low 

.500 

.079 

.056 

2 

Brors  Fter  CP  Attempt,  VE2,  Ftigh 

.460 

.246 

.142 

3 

Brors  Fter  CP  Attempt,  VE2,  Low 

.508 

.090 

.063 

2 

Table  1.  Errors  per  CP  Attempted  by  Group  and  GZ  Ability 


Table  2  shows  the  descriptive  statistics  concerning  the  total  error  score  per  CP 
attempted.  Again,  no  significant  differences  can  be  inferred.  In  this  case,  it  is  not  readily 
apparent  if  any  one  style  of  training  technique  minimizes  the  differences  between  high  and 
low  GZ  ability.  The  largest  percentage  difference  is  in  the  real  world  group,  suggesting 
that  while  direct  exploration  for  a  limited  period  may  be  effective  for  high  GZ  subjects, 
low  ability  subjects  would  best  avoid  this  technique.  The  subjects  in  the  Goerger  study  had 
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the  largest  differences  between  the  ability  levels.  The  extended  study  time  may  explain  the 
closer  performance  levels  for  the  train-to-standard  groups  (VEl,  MAPI),  and  the  poor 
map  and  resulting  easier  route  selection  may  do  the  same  for  the  low  fidelity  map  groups 
(MAP2,  VE2).  In  these  groups,  there  seemed  little  difference  in  how  quickly  the  high  and 
low  spatial  ability  subjects  realized  they  had  made  an  error.  Considering  only  the  Goerger 
groups,  while  as  a  whole  the  map  group  realized  their  errors  quicker,  the  difference 
between  the  spatial  ability  groups  as  a  percentage  was  the  smallest  in  the  VE  group. 


Descriptive  Statistics 
Split  By:  Group,  Abiiity  Group 


Mean 

Std.  Dev. 

Std.  Bror 

Count 

Total  &ror  Score/Attempt,  Total 

58.327 

68.549 

11.587 

35 

Total  Bror  Score/Attennpt,  MAP,  Hgh 

35.815 

31.896 

18.415 

3 

Total  Bror  Score/Attempt,  MAP,  Low 

153.871 

68.507 

48.442 

2 

Total  Bror  Score/Attennpt,  MAPI,  Hgh 

25.558 

15.972 

11.294 

2 

Total  Bror  Score/Attempt,  MAPI,  Low 

26.602 

13.449 

7.765 

3 

Total  Bror  Score/Attenpt,  MAR2,  Hgh 

26.200 

8.084 

4,042 

4 

Total  Bror  Score/Attennpt,  MAR2,  Low 

17.097 

• 

• 

1 

Total  Bror  Score/Attennpt,  RW,  Hgh 

17.352 

1.074 

.759 

2 

Total  Bror  Score/Attempt,  RW,  Low 

121.842 

60.342 

34.839 

3 

Total  Bror  Score/Attennpt,  VE,  Hgh 

59.066 

29.893 

21.138 

2 

Total  Bror  Score/Attennpt,  VE,  Low 

180.597 

148.275 

85,606 

3 

Total  Bror  Score/Attempt,  VE1 ,  Hgh 

23.935 

12.752 

7.363 

3 

Total  Bror  Score/Attempt,  VE1,  Low 

30.081 

8.717 

6.164 

2 

Total  Bror  Score/Attempt,  VE2,  Hgh 

36.325 

8.575 

4.951 

3 

Total  Bror  Score/Attempt,  VE2,  Low 

36.181 

29,443 

20.819 

2 

Table  2.  Total  Error  Score/ Attempt  by  Group  and  GZ  Ability 


Overall,  the  results  in  this  section  suggest  that  it  may  be  beneficial  for  low  spatial 
ability  subjects  to  train  using  the  VE,  as  this  may  help  to  visualize  the  map  in  a  way  that 
they  have  difficulty  doing  on  their  own.  Further  research  into  this  area  is  needed  to 
determine  if  low  ability  subjects  really  would  be  better  training  by  one  particular  method. 

8.  Non-Error  Map  Checks 

During  the  course  of  the  orienteering  experiment,  individuals  would  sometimes 
conduct  map  checks  even  if  they  were  stiU  on  their  planned  path  or  at  a  known  point. 
These  non-error  map  checks  are  referred  to  as  maintenance  checks.  Some  of  the 
individuals  may  have  been  disoriented  at  the  time  of  the  check  (even  if  not  yet  in  error). 
Most,  however,  used  these  checks  to  confirm  their  memory  of  their  planned  route  or  to 
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refresh  themselves  as  to  the  features  they  planned  to  use  to  assist  them  in  finding  the  next 
control.  Some  subjects  had  planned  to  request  the  compass  at  certain  locations  to  conduct 
dead  reckoning  from  a  known  attack  point.  As  this  style  of  map  check  is  completely 
different  from  those  caused  by  errors,  the  data  was  analyzed  by  considering  only  the 
maintenance  map  checks  to  see  what  effect,  if  any,  the  training  group,  user’s  spatial 
ability,  user’s  navigational  ability,  etc.,  had  on  their  requesting  maintenance  checks.  Figure 
4.40  shows  the  box  plot  results  for  the  training  groups. 
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Figure  4.40.  Box  Plot  of  Maintenance  Checks  (Group) 

The  analysis  of  means  is  not  statistically  significant,  F(6,35)  =  2.038,  P  =  0.937 
Most  groups  averaged  around  three  points  worth  of  maintenance  map  checks.  Direct 
observation  of  the  results  suggests  that  the  train-to-standard  VE  group  (VEl)  averaged 
the  least  number  of  maintenance  checks  (0.7).  This  suggests  that  extended  exploration  of 
the  environment  in  the  VE  allowed  them  to  better  internalize  their  route  so  that  they 
needed  fewer  reminders  during  the  course.  The  low  fidelity  map-only  group  also  scored 
below  the  average.  It  is  likely  that  the  lack  of  information  on  the  map  persuaded  subjects 
not  to  consult  it. 

Examimng  the  results  based  on  ability  group,  there  is  no  discemable  differences 
between  the  number  of  maintenance  map  checks  and  either  GZ  ability  level,  Santa  Barbara 
level,  or  the  user’s  experience.  Furthermore,  regression  analysis  showed  no  association 
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between  the  number  of  maintenance  checks  and  the  subject’s  resulting  landmark  score 
(F(l,35)  =  0.550,  P  =  0.4637).  Figure  4.41  shows  this  regression  analysis. 


Regression  Plot 


Figure  4.41.  Regression  Plot  of  Maintenance  Checks  vs.  Landmark  Score 

Figure  4.42  shows  the  same  analysis  this  time  comparing  maintenance  checks  and 
the  errors  per  control  attempted.  Again,  there  is  no  correlation  between  the  number  of 
maintenance  checks  and  the  subject’s  performance. 


Regression  Plot 


Figure  4.42.  Regression  Plot  of  Maintenance  Checks  vs.  Errors  per  CP  Attempted 

From  the  above,  we  can  conclude  that  while  the  training  group  may  have  an  effect 
on  the  number  of  maintenance  checks  a  subject  takes,  the  number  of  maintenance  checks  a 
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subject  takes  does  not  appear  to  affect  their  actual  performance.  The  number  of 
maintenance  checks  appears  totally  independent  from  the  subject’s  ability  level. 

9.  Correlation  Between  Disorientation  in  Virtual  Environment  and 
Disorientation  in  Real  World 

In  his  1998  study,  Goerger  noticed  a  possible  correlation  between  subject’s 
disorientation  in  the  VE  and  in  the  same  location  in  the  real  world  [GOER  98b].  The  same 
general  effect  was  noticed  on  several  occasions  in  this  experiment  as  well.  Subject  VE2-1 
had  trouble  in  the  VE  looking  for  CPS.  The  subject  in  fact  bypassed  it  on  the  left  and 
continued  into  the  depression  near  CP4  before  realizing  the  error.  Despite  resolving  this  in 
the  VE,  the  subject  made  the  exact  same  error  while  running  the  course,  even  remarking 
“Shoot,  I  did  the  same  thing  in  the  computer”.  No  concrete  conclusion  can  be  drawn, 
however,  because  of  the  limited  number  of  participants  who  demonstrated  this  effect. 

Generally,  subjects  were  asked  which  point  they  found  to  be  the  hardest  to  find  in 
the  VE  and  which  they  felt  would  be  the  toughest  in  the  real  world.  Some  of  the  subject’s 
difficulties  came  from  the  limitations  in  the  VE  model  (cannot  see  through/under  brush). 
Many  subjects  assumed  CP5  would  be  difficult  to  find  based  on  their  model  experience. 
Several  subjects  also  were  very  up  front  about  their  planned  usage  of  trail  intersections  for 
navigation.  Many  commented  that  if  they  could  not  discern  those  intersections  on  the 
actual  course,  they  would  be  m  trouble. 

Several  subjects  got  into  trouble  by  planning  cross-country  movements  using  the 
VE.  During  the  course  of  the  VE  training,  they  rehed  heavily  on  the  conpass  to  cut  direct 
movement  while  “beating  brush.”  The  ease  of  this  n^thod  in  the  VE  disarmed  the  subjects 
to  the  actual  difficulty  of  their  planned  route,  and  the  subjects  had  difficulty  replicating 
these  routes  on  the  actual  course. 

10.  Resolving  Ambiguities  in  Mental  Maps 

One  of  the  biggest  difficulties  facing  the  subjects  was  resolving  differences  in  their 
mental  maps  upon  arrival  on  the  ground.  These  difficulties  were  noted  by  Goerger  in  his 
experiment,  and  similar  effects  were  observed  during  this  experiment.  Goerger  felt  that 
each  group  (map-only,  VE  and  real-world)  faced  it’s  own  specialized  challenges  in  this 
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regard.  These  differences  are  partly  due  to  the  way  the  information  bout  the  environment 
was  acquired. 

VE  (and  real-world)  subjects  gathered  much  of  their  information  from  dynamic 
imagery.  Many  VE  subjects  described  it  as  playing  back  a  movie  in  their  heads.  They 
would  try  to  match  the  imagery  their  eyes  were  seeing  to  this  mental  movie  in  their  heads 
to  determine  if  they  were  on  course.  The  potential  pitfall  in  this,  as  discussed  by  Goerger 
[GOER  98b],  is  when  this  mental  movie  is  incomplete  or  inaccurate.  Then  users,  as 
opposed  to  confirming  their  location,  merely  become  confused  and  disoriented. 

Map-only  subjects  tended  to  store  their  information  as  a  list  of  objects  or  actions 
referred  to  as  static  imagery  or  symbolic  imagery.  This  gave  them  knowledge  about 
particular  points  of  interest  but  fuzzier  information  about  indistinct  areas.  This  in  turn 
lends  the  map-only  groups  more  slack  when  conducting  the  course,  as  they  are  only 
looking  for  a  few  key  points  of  interest.  Map-only  subjects  became  confused  only  when 
they  encountered  key  structures  out  of  turn  or  encountered  areas  that  appeared  similar  to 
their  focus  area  (see  Chapter  III  Errors).  Many  map-only  subjects  discarded  distance 
estimation  entirely,  relying  solely  upon  their  mental  checklist.  While  this  may  not  have 
assisted  them,  it  did  not  hinder  their  completion  of  the  course  unless  they  later  tried  to 
apply  distance  estimation  to  their  mental  model.  Subjects  who  stuck  to  a  poor  strategy 
performed  better  than  those  who  tried  to  change  tactics  “in  the  middle.”  This  later  group 
usually  only  further  confused  themselves  and  ended  up  becoming  totally  disoriented. 

For  all  of  the  groups,  it  was  critical  that  they  quickly  adjust  their  mental  models  to 
the  terrain.  This  meant  transferring  the  picture/movie  in  their  minds  into  actual  distances 
covered  on  the  ground.  Once  subjects  established  the  distance/scale  relationships,  they 
usually  had  little  difficulties  later.  The  caveat  is  that  subjects  with  irq)recise  mental  models 
often  radically  change  their  entire  model,  not  just  the  bad  portions.  One  map  subject  who 
did  not  memorize  distances,  underestimated  the  distance  between  the  SP  and  CPI,  as  well 
as  CPI  and  CP2.  Having  discovered  that  these  points  were  much  closer  than  he  expected, 
he  radically  shorted  his  mental  map  image  of  the  area.  This  resulted  in  his  stopping  two 
hundred  meters  short  of  CP4  when  he  first  initiated  his  search.  Because  the  first  two 
controls  had  been  closer  than  he  thought,  he  guessed  that  the  whole  course  was  that  way. 
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The  VE  group  in  particular  had  the  most  trouble  in  quickly  resolving  differences 
between  the  real  world  and  their  mental  model.  The  first  movement  in  the  course,  from  the 
SP  to  CPI,  was  where  VEl  subjects  averaged  their  greatest  amount  of  error  distance. 
Figure  4.43  shows  the  distance  per  error  for  leg  one  for  all  groups.  The  lack  of  an 
effective  means  of  discerning  scale  and  distance  traveled  in  the  VE  likely  contributed  to 
this  early  difficulty. 


Box  Plot 
Split  By:  Group 


Figure  4.43.  Distance  per  Error  SP  to  CPI  (Group) 


While  no  statistical  significance  can  be  drawn,  direct  observations  show  that  the 
real-world  group  made  the  least  error  distance  on  this  movement.  Also,  disregarding  one 
Goerger  VE  subject  who  circled  the  course  area  looking  for  CPI,  aU  of  the  VE  groups  on 
this  leg  performed  about  the  same  regardless  of  training  method.  Once  the  mental 
ambiguities  were  resolved,  however,  the  train-to-standard  VE  group  was  able  to  perform 
better  on  the  rest  of  the  course. 

11.  Map  Reading 

Goerger  did  not  determine  any  difference  in  performance  based  on  map  reading 
ability.  The  results  in  this  experiment  did  find  some  noticeable  effects  on  performance 
based  on  map  reading  ability.  For  a  subject’s  landmark  scores,  the  map  reading  score 
showed  a  significant  effect,  with  F(l,34)  =  7.143,  P  =  0.0116.  Figure  4.44  show  the 
regression  plot. 
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Regression  Plot 


Figure  4.44.  Regression  Plot  of  Landmark  Score  vs.  Map  Reading  Score 

Observation  of  the  data  suggests  that  while  a  high  map  reading  score  is  not  a  good 
predictor  of  success,  a  low  map  reading  score  is  a  good  predictor  of  failure.  Those 
subjects  who  had  difficulty  with  basic  map  reading  skills  in  turn  had  difficulty  in  locating 
the  control  points. 

Direct  observation  of  the  data  suggested  that  subjects  with  poorer  map  reading 
skills  also  tended  to  make  more  errors  per  CP  attempted  and  traveled  more  distance  per 
error.  This  may  be  because  they  cannot  identify  viable  routes,  or  the  necessary  features  to 
help  keep  them  on  their  route.  Neither  regression  was  significant  statistically  however  (for 
the  errors  per  CP  attempted  F=  3.916,  P  =  0.0562,  distance  per  error  F  =  2.701,  P  = 
0. 1098).  The  regressions  for  both  are  shown  in  Figure  4.45  below. 


Figure  4.45.  Regression  Plots  of  Map  Reading  vs.  Total  Error  Score/Attenpt  and 
Map  Reading  vs.  Errors  per  CP  Attempt 
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A  subject’s  map  reading  skills  also  may  have  had  an  effect  on  the  difficulty  of  the 
route  they  chose.  While  no  statistically  valid  difference  exists,  direct  observation  (see 
Figure  4.46)  of  the  regression  comparison  suggests  that  subjects  with  higher  map  reading 
skills  choose  less  difficult  routes. 


Map  Reading  Test 


Y  =2.892-. 067 *X;  R^2  =  .061 

Figure  4.46.  Regression  Plot  of  Lisp  Route  Difficulty  vs.  Map  Reading  Score 


The  map  reading  scores  were  examined  to  see  how  much  they  were  influenced  by 
the  subject’s  individual  differences.  There  were  no  significant  differences  between  the  map 
reading  scores  of  high  and  low  GZ  ability  subjects.  This  is  shown  in  Figure  4.47. 


Box  Plot 

Split  By:  Ability  Group 


Map  Reading  Test 

Figure  4.47.  Map  Reading  Score  (GZ  Ability  Group) 
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There  was  also  no  difference  between  the  Santa  Barbara  ability  groups.  There  was 
an  observed  difference  based  On  the  bar-line  evaluation.  While  those  classified  as  experts 
did  not  outperform  the  others,  those  classified  as  beginners  did  demonstrate  weaker  map 
reading  skills.  Since  map  reading  is  largely  a  learned  skill,  this  is  to  be  expected.  The  data 
is  shown  in  Figure  4.48.  ' 

Box  Plot 

Split  By:  Self*Bfal 


Map  Reading 


Figure  4.48.  Map  Reading  Score  (Self-Eval) 


Examining  the  distribution  of  map-reading  scores  across  the  groups,  we  find  that 
the  lowest  scoring  map-readers  were  in  the  original  VE  group.  See  Figure  4.49.  This 
brings  up  an  interesting  question  of  whether  or  not  the  subject  scored  low  because  he 
could  not  read  a  map,  or  because  of  the  training  tool  he  used?  The  data  may  have  to  be 
analyzed  using  MANOVAs  to  answer  which  criteria  had  the  strongest  effect  on 


Figure  4.49.  Map  Reading  Score  (Group). 
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V.  CONCLUSIONS  AND  RECOMMENDATIONS 


A.  CONCLUSIONS 

1.  C^neral  Conclusions 

The  thesis  experiments  studied  the  effects  of  various  training  methods  on 
the  transfer  of  spatial  knowledge  to  a  real  world  environment.  The  first  experiment  used  a 
high-resolution  1:5,000  scale  map  and  the  high  fidelity  real  time  VE.  The  subjects  either 
trained  only  using  the  map  or  with  the  map  and  VE  together.  Subjects  had  to  meet  a  set 
standard  before  continuing  on  to  the  actual  testing  phase.  The  second  experiment  used  a 
lower  fidelity  1:24,000  scale  map  but  the  same  high  fidelity  VE.  Subjects  were  again  either 
map-only  or  VE  and  map  together.  Training  time  was  limited  to  exactly  one  hour.  The 
results  of  the  experiments  were  then  analyzed  against  each  other  and  the  original  Goerger 
results  [GOER  98b].  The  following  conclusions  are  drawn  fi-om  both  the  quantitative  and 
qualitative  results  previously  presented: 

a.  Subjects  who  trained  to  standard  with  a  high  fidelity  VE 
demonstrated  superior  route  and  landmark  knowledge  to  any  other  training  group 
(Chapter  IV,  Sections  A.4  and  A.5). 

b.  Spatial  Ability  (including  Santa  Barbara  level  or  self-assessment 
of  spatial  ability)  plays  a  significant  role  in  an  individual’s  ability  to  obtain  and 
demonstrate  spatial  knowledge  (Chapter  IV,  Sections  B.4  and  B.5).  High  spatial 
ability  subjects  perform  better  on  route  and  landmark  tasks  than  do  low  spatial 
ability  subjects. 

c.  Subjects  with  poor  map  reading  skills  are  likely  to  have 
difficulty  in  navigational  tasks  and  demonstrate  significantly  poorer  spatial 
knowledge  than  any  other  subjects  (Chapter  IV,  Section  B.ll). 

d.  The  difficulty  of  the  planned  route  has  an  effect  on  its  successful 
completion,  and  a  significant  effect  of  the  amount  of  spatial  knowledge  gained 
(Chapter  IV,  Section  B.2).  The  easier  the  route,  the  better  the  overall 
perfi'omance. 

e.  Adjusting  the  fidelity  of  the  map  causes  subjects  to  adjust  their 
plaimed  routes  to  the  information  that  is  provided  (Chapter  IV,  Section  B.l).  The 
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accuracy  of  the  map  in  representing  terrain  can  have  a  dramatic  effect  on  errors  at 
specific  locations  (Chapter  IV,  Section  B.l). 

2.  Performance  by  Study  Group 

Based  on  the  outcomes  of  the  three  experiments,  the  results  suggest  that 
those  who  trained  to  a  standard  and  not  to  time,  gained  more  spatial  knowledge  about  a 
one  km  square  piece  of  terrain  than  those  who  trained  to  a  one-hour  time  limit.  VE 
subjects  who  trained  to  standard  had  the  best  overall  landmark  and  route  knowledge 
performance.  Subjects  who  were  given  only  one  hour  to  train  and  a  high  fidelity  map 
performed  better  than  those  with  only  an  hour,  the  high  fidehty  map,  and  the  high  fidelity 
VE  [GOER  98b].  Results  suggest  that  with  only  one  hour  of  training  and  a  less  detailed 
map  (but  with  one  critical  area  being  more  accurate),  the  subjects  would  adjust  then- 
routes  to  the  detail  provided.  They  then  performed  better  in  followmg  these  simple  routes 
than  those  who  planned  their  routes  on  the  more  detailed  map. 

3.  Performance  by  Spatial  Ability 

The  results  suggest  that  regardless  of  training  group,  individuals  with 
above  average  spatial  ability  scores  (as  determined  by  the  Guilford-Zimmerman  test) 
demonstrated  more  spatial  knowledge  than  individuals  with  low  spatial  ability  scores. 
Subjects  with  high  Santa  Barbara  ability  scores  also  outperformed  those  with  low  Santa 
Barbara  ability  scores.  Higher  ability  level  individuals  selected  easier  routes  and  executed 
them  better.  The  results  suggest  that  high  ability  individuals  had  better  egocentric 
knowledge  than  their  counterparts.  High  spatial  ability  individuals  demonstrated  slightly 
better  exocentric  knowledge  (survey  knowledge)  than  low  ability  individuals.  Using  the 
train-to-standard  VE  seemed  to  minimize  the  differences  between  spatial  ability  users 
(Chapter  IV,  Section  B.7). 

4.  Performance  by  User  Experience/Map  Reading  Skills 

Users  who  classified  themselves  as  expert  navigators  made  fewer  errors 
than  intermediate  or  beginner  level  subjects  did.  They  also  conducted  fewer  map  checks. 
While  they  did  find  slightly  more  controls,  this  was  not  significant.  There  was  no 
significant  difference  between  ability  levels  in  how  fast  subjects  recognized  their  errors. 
Nor  did  subjects  demonstrate  differences  in  survey  knowledge  based  on  their  experience 
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classification.  Beginners  were  much  less  likely  to  make  map  checks,  because  they  did  not 
realize  they  needed  them  or  were  uncertain  as  to  their  benefit.  Beginners  also  had  much 
lower  map  reading  skills  on  average. 

While  a  high  map  reading  score  was  not  a  good  predictor  of  success,  a  low  map 
reading  score  was  an  excellent  predictor  of  failure.  Subjects  with  poor  map  reading  skills 
had  difficulty  planning  executable  routes,  had  problems  locating  the  controls,  were  more 
apt  to  make  errors,  and  were  less  likely  to  recover  quickly  fi:om  those  errors. 

5.  Route  Selection 

Subjects  who  planned  simpler  routes  made  fewer  errors  and  found  more 
controls  than  those  who  planned  harder  routes.  These  simpler  routes  included  more  key 
features  that  could  be  used  as  reference  points  or  navigational  aids.  This  helped  them 
recover  fi'om  errors  they  did  make  faster  than  other  subjects.  They  avoided  areas  where 
parallel  errors  were  likely. 

6.  Map  Fidelity 

Subjects  who  were  given  the  low  fidelity  map  adjusted  by  planning  much 
simpler  routes  in  general  and  by  sticking  to  the  boundary  roads  whenever  possible.  The 
importance  of  accuracy  in  the  map  was  highlighted  in  the  improved  results  in  finding  CP4. 
The  original  Banker  map,  however  detailed,  was  not  clear  in  showing  this  depression, 
which  caused  more  errors  for  those  who  relied  on  that  map.  Subjects  who  had  access  to 
highly  detailed  maps  and  had  time  constraints  gamed  more  spatial  knowledge  than  their 
VE  counterparts  [GOER  98b]. 

B.  SIGNIFICANCE 

1.  Study  Group 

If  mission  constraints  include  limited  preparation  time  and  highly  detailed 
maps,  individuals  who  only  use  maps  will  gain  more  information  about  the  mission  area 
than  those  who  also  use  the  VE.  If  time  is  not  a  constraint,  individuals  who  use  the  VE 
will  gam  more  information  than  those  who  only  study  the  map.  Individuals  should  tailor 
their  learning  method  based  on  the  time  available.  If  there  is  not  enough  time  to 
thoroughly  explore  the  VE  and  make  multiple  iterations,  then  stay  with  the  map.  This 
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basic  rule  will  help  individuals  focus  their  efforts  where  they  will  make  the  most  gain.  If 
time  is  limited  and  only  poor  maps  are  available,  using  the  VE  may  also  prove  valuable  as 
long  as  there  is  information  in  the  VE  that  is  not  included  on  the  map. 

2.  Spatial  Ability 

High  spatial  ability  individuals  are  better  suited  as  navigators  than  low 
spatial  ability  individuals.  These  individuals  should  be  identified  if  possible  and  be  assigned 
in  positions  requiring  navigational  skill.  If  it  is  possible  to  improve  an  individual’s  spatial 
ability,  it  should  be  done.  Low  spatial  ability  individuals  may  have  an  easier  time  using  the 
VE  in  training  than  just  the  map  alone.  This  may  be  due  to  their  difficulty  in  mentally 
orientating  and  visualizing  a  map. 

3.  User  Experience 

Experienced  navigators  make  less  errors  and  are  more  confident,  if  not 
actually  better,  at  navigational  tasks.  The  only  way  to  become  experienced  is  to  practice. 
Individuals  must  conduct  regular  navigational  practice  across  a  variety  of  terrain  types  and 
using  all  navigational  techniques  to  stay  proficient.  Navigation  is  best  learned  by  doing. 
Individuals  may  also  get  involved  in  recreational  activities  that  utilize  navigational  sVills 
(Sport  Orienteering,  Scouting)  to  practice  and  have  fun  at  the  same  time. 

Map  reading  is  a  critical  and  perishable  skill.  Individuals  who  cannot  read  a  map 
will  have  difficulty  performing  even  the  most  basic  of  navigational  tasks.  It  is  for  this 
reason  that  basic  map  reading  is  the  first  navigational  skill  taught  to  soldiers.  Individuals 
should  plan  to  refi-esh  their  skill  in  this  area  prior  to  missions. 

4.  Route  Selection 

Good  route  selection  is  critical  to  navigational  performance.  Individuals 
who  plan  routes  with  numerous  easy-to-remember  reference  points  or  siiiq)lified  paths  are 
much  more  likely  to  execute  them.  Mission  constraints  may  prohibit  the  use  of  the  most 
direct  or  the  easiest  route.  Individuals  should  try  to  tailor  their  route  difficulty  to  the 
mission.  Difficulty  for  difficulty’s  sake  is  not  desirable.  A  good  route  should  include  many 
of  the  navigational  aids  described  in  Appendix  P. 
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5.  Map  Fidelity 

Maps,  regardless  of  how  detailed,  must  accurately  show  the  terrain  in  a 
clear  manner.  A  map  with  too  much  detail  clutter  may  actually  induce  error  at  certain 
locations.  An  accurate  high  fidelity  map  can  be  relied  upon  for  success  if  time  is  limited.  A 
low  fidelity  map  will  force  subjects  to  siiiq)lify  their  planned  movements  to  use  the 
information  provided  by  the  map.  A  high  fidelity  map  may  offer  users  more  freedom  as 
they  can  have  multiple  reference  points  to  choose  from  when  planning  their  route. 

Ideally,  a  high  fidelity  VE  has  the  same  information  as  a  high  fidehty  map.  The  VE 
takes  longer  to  build  and  is  not  as  easy  to  reproduce  nor  can  it  be  carried  with  you.  Thus 
regardless  of  the  time  allowed  for  preparation,  a  map  should  always  be  included  in  the 
program. 

C.  FUTURE  WORK 

While  this  thesis  may  have  vahdated  the  utility  of  using  a  VE  to  transfer  spatial 
knowledge  about  a  natural  area,  there  remain  significant  areas  for  future  work  and 
exploration. 

1.  Immersive  vs.  Nonlmmersive 

Ruddle,  Randle,  Payne,  and  Jones  [RUDD  96]  concluded  that  there  was  no 
significant  difference  between  immersive  and  non-immersive  VEs  with  regards  to 
performance  results.  However,  their  experiment  was  conducted  in  an  indoor  environment 
that  was  relatively  small  in  scale.  Will  the  same  conclusion  hold  to  a  large  outdoor 
environment?  Most  VE  subjects,  judging  by  their  survey  answers,  felt  comfortable  with 
the  three  screen  wide  FOV  display.  However,  this  may  not  be  the  most  immersive,  nor  the 
best  way  to  display  the  VE.  Future  experiments  can  compare  the  current  VE  results  to 
those  obtained  using  a  more  immersive  interface.  The  MOVES  curriculum  at  NPS  is 
currently  building  their  own  VE  CAVE  that  could  be  used  in  the  experiment  in  addition  to 
head  mounted  displays.  The  effect  of  downsizing  the  wide  screen  TVs  to  a  more 
affordable  and  space  friendly  three  monitor  configuration  should  also  be  studied. 
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2.  Incoiporating  Sense  of  Scale  into  the  VE 

One  comment  heard  repeatedly  from  the  VE  subjects  was  how  they  lacked  a  real 
sense  of  scale  in  the  VE  model.  They  complained  of  being  unable  to  accurately  estimate 
distance  and  often  felt  uncomfortable  when  first  introduced  into  the  real  world  until  they 
were  able  to  focus  their  mental  map.  This  result  backs  up  the  findings  of  Witmer  and  Kline 
[WITM  98]  in  that  judging  distance  in  a  VE  is  difficult,  and  usually  underestimated. 
Goerger  had  originally  added  two  HMMWVs  to  the  model  at  the  start  point  in  order  to 
give  a  sense  of  scale  [GOER  98b],  but  most  subjects  didn't  find  this  particularly  useful. 
There  are  several  possible  options  for  introducing  a  sense  of  scale  and  distance  traveled 
into  the  model.  It  would  be  possible  to  incorporate  some  sort  of  sound  cue  that  could  beep 
after  a  set  distance  of  travel.  A  visual  pace  count  or  pedometer  could  also  be  displayed. 
Users  should  be  able  to  reset  this  counter  easily  as  they  maneuver  through  the 
environment.  Another  option  would  be  to  change  the  interface  into  one  that  actually 
requires  locomotion  by  the  subject,  whether  by  treadmill  or  cycle.  While  these  last  two 
methods  may  negate  the  advantage  of  being  able  to  explore  faster  than  humanly  possible, 
it  would  allow  users  a  more  realistic  sense  of  traveling  and  to  acquire  a  sense  of  distance 
traveled. 

3.  Fidelity  Levels 

The  current  model  is  classified  as  high  fidelity  (see  Chapter  III,  Section  C  Model). 
Now  that  the  transfer  of  spatial  knowledge  from  VE  to  natural  terrain  is  a  proven  concept, 
new  experiments  could  adjust  the  fidelity  level  of  the  model  to  determine  what  is  both  the 
optimal  level  of  fidelity  for  transfer,  and  what  is  the  minimum  level  of  fidelity  to  achieve 
this  transfer.  Hopefully,  reducing  the  fidelity  would  enable  the  model  to  be  run  on  a  less 
powerful  machine.  By  determining  the  optimal  levels,  we  can  ensure  we  develop  models 
that  reach  this  but  do  not  go  past  the  point  of  diminishing  returns.  The  model  currently 
uses  digital  photographs  to  create  photo-realistic  bushes,  trees,  and  buildings.  However, 
all  this  comes  at  the  expense  of  processing  power.  What  if  the  same  training  effect  can  be 
achieved  by  substituting  green  cones  for  trees?  Is  it  really  an  advantage  to  exactly  replicate 
the  buildings  with  detailed  textured  photographs  or  would  a  sinple  black  square  be 
enough?  Obviously  the  idea  of  just  incorporating  symbolic  representation  (as  is  done  with 
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a  map)  may  make  the  model  appear  less  "realistic",  but  would  that  necessarily  lessen  the 
training  in5)act? 

4.  Transfer  to  a  Desk-Top  Model 

The  current  model  requires  a  high-end  multi-processor  machine  to  run.  This 
machine  is  both  expensive  and  takes  up  considerable  space.  It  would  be  of  tremendous 
interest  to  the  Army  to  determine  if  the  model  could  be  ported  to  a  smaller  desktop 
machine.  Obviously,  this  would  result  in  changes  to  both  the  display  and  to  the  model 
fidelity.  However,  these  reductions  would  be  accompanied  by  a  cost  and  space  savings.  If 
such  a  transfer  were  proven  feasible,  it  would  be  a  boon  to  the  military  and  would  allow 
for  a  more  widespread  distribution  of  a  VR  training  system. 

5.  Design  a  MOUT  Environment 

It  has  been  shown  that  participants  can  gain  spatial  knowledge  of  indoor  man¬ 
made  environments,  and  in  natural  outdoor  areas.  The  next  logical  step  is  to  explore  the 
knowledge  transfer  in  a  combination  of  these  environments,  namely  urban  areas.  The 
perfect  setting  for  this  is  available  locally  with  the  old  MOUT  site  (Military  Operations  in 
Urban  Terrain)  on  Fort  Ord.  This  site  consists  of  a  several  mock  city  blocks  with  multiple 
story  cinderblock  buildings  with  rooms,  windows,  staircases,  and  sewers.  The  area  has 
streets  and  parks  as  weU.  The  military  used  the  site  to  practice  fighting  in  cities  and  other 
built  up  areas.  The  area  is  currently  used  almost  daily  by  local  and  national  law 
enforcement  agencies  to  train  quick  reaction  forces  or  SWAT  teams  [BELC  99].  The 
military  uses  a  standard  design  for  its  MOUT  training  areas,  so  detailed  blueprints  should 
be  readily  available  to  facilitate  accurate  model  construction.  Given  the  high  usage  of  the 
area  by  local  law  enforcement,  a  large  pool  of  test  subjects  is  potentially  available.  It 
should  be  possible  to  arrange  for  test  subjects  and  develop  the  standards  for  evaluating  the 
models' effectiveness  to  everyone's  satisfaction. 

6.  Make  the  Model  More  3D/  More  Realistic 

At  a  recent  virtual  reality  conference  held  at  NFS,  many  psychologists  complained 
about  the  use  of  billboard  photos  for  trees  or  other  non-flat  objects.  They  said  this 
technique  robs  the  environment  of  its  three-dimensional  feel  and  makes  it  seem  less  real. 
Coupled  with  the  fidelity  research  above,  future  work  could  focus  on  making  the  trees  and 
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other  objects  seem  less  flat  and  more  solid.  Another  problem  that  presented  itself  in  the 
model  was  the  lack  of  the  ability  to  raise  or  lower  the  avatar's  viewpoint  from  the  set  5'8’' 
height.  While  the  15-meter  pop-up  view  was  provided,  this  still  did  not  allow  the  user  the 
flexibility  that  they  possess  in  the  real  world.  Often  when  searching  for  a  point  in  the  VE, 
subjects  desired  to  look  under  branches  but  could  not  lower  themselves  down  without  just 
backing  up.  But,  in  dense  areas,  backing  up  would  usually  put  a  new  tree  in  front  of  them 
further  obscuring  the  view.  The  ability  to  lower  the  avatar's  viewpoint  might  allow  for 
more  "realistic"  training.  A  third  area  for  work  is  in  collision  detection.  The  current  model 
has  no  collision  detection  and  allows  users  to  walk  unimpeded  through  the  trees  and 
buildings.  One  potential  problem  with  this,  seen  in  a  couple  of  VE  subjects,  was  the 
selection  of  routes  based  on  this  "easy"  movement.  Upon  arrival  into  the  real  world,  these 
subjects  regretted  choosing  a  cross-country  route  through  dense  brush  that  was  easy  in  the 
model  but  difflcult  and  time-consuming  in  the  real  world  (see  Figure  5.1  Tight  Squeeze). 
Adding  some  code  to  the  model  to  slow  movement  through  difficult  terrain  (whether 
dense  woods,  swamps,  etc.)  would  make  exploration  in  the  model  more  realistic.  It  would 
also  make  it  possible  to  allow  weather  effects  to  be  incorporated  into  the  model  later  (such 
as  slowed  movement  from  mud  or  snow). 


Figure  5.1.  Tight  Squeeze 
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7.  Cooperative  Learning 

Within  the  educational  community,  many  groups  are  reporting  that  computer 
trainers  work  best  when  cooperative  learning  is  incorporated  into  the  learning  program. 
The  optimal  size  seems  to  be  two  to  three  students  [ROSE  95].  Rose  concludes  that 
“...virtual  reality  technology  that  fosters  collaboration  will  yield  even  greater  educational 
bene£its”[Rose  95].  The  effect  of  cooperative  learning  could  be  done  so  two  subjects 
could  conduct  the  training  phase  together  but  then  be  tested  in  the  environment  separately 
or  together.  The  results  would  be  to  see  if  they  performed  better  as  individuals  than  those 
who  trained  alone.  This  would  also  be  more  realistic  because  in  the  active  military  most 
navigation  is  done  by  a  group,  not  an  individual.  Individuals  would  pool  knowledge  to 
arrive  at  a  more  informed  decision.  Obviously,  introducing  cooperative  learning  would 
also  introduce  a  host  of  other  factors,  not  the  least  of  which  include  individual  personality, 
compatibility,  and  cooperation. 

8.  Training  Navigation  as  a  Secondary  Task 

Land  navigation  is  never  the  end  goal  of  any  military  navigation  other  than  in 
training  courses.  However,  it  is  an  essential  skill  to  the  successful  completion  of  most 
missions.  Given  its  critical,  but  secondary,  nature,  it  would  be  interesting  to  see  if  spatial 
knowledge  could  be  obtained  while  ostensibly  not  training  navigation  but  another  task.  An 
ideal  experiment  could  use  a  popular  3D-shooter  type  game  such  as  Quake  or  Doom 
Subjects  would  be  told  they  are  conducting  research  in  virtual  environments,  not  in  land 
navigation  in  particular.  They  would  be  presented  with  the  game  environment  built  to 
replicate  the  outdoor  Fort  Ord  area.  The  players  would  be  given  limited  ammunition  and 
have  the  goal  of  killing  all  the  enemies.  At  each  control  point,  additional  ammo  is  found, 
as  well  as  one  or  more  enemies.  However,  to  encourage  the  players  to  visit  the  controls  in 
order,  the  program  spawns  additional  enemies  at  the  controls  based  on  a  time  factor,  with 
more  enemies  being  produced  at  the  lower  number  controls  quicker  than  at  the  higher 
number.  Once  a  control  is  captured  however,  it  stops  producing  enemies.  In  this  manner 
players  would  optimally  clear  the  lower  controls.  Otherwise  they  may  be  swanq^d  with 
enemies  and  be  unable  to  "win".  After  several  sessions,  the  players  would  be  taken  to  the 
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outdoor  enviromnent  and  tested  on  the  navigational  tasks  to  see  what  knowledge  was 
transferred  during  the  course  of  their  gaming. 

9.  Evaluating  Navigation  as  a  Secondary  Task 

As  with  Section  8  above,  land  navigation  rarely  stands  alone.  To  provide  validity 
of  the  model’s  effectiveness,  the  evaluation  phase  should  incorporate  some  secondary  task 
that  requires  constant  thinking,  but  is  not  totally  overwhelming.  A  possible  solution  is  the 
introduction  of  a  hunter.  As  with  most  military  mission,  there  is  an  enemy  to  be  avoided. 
This  person  is  armed  with  a  paintball  gun  and  is  actively  searching  for  the  subject.  Our 
subject  is  told  that  person  has  a  map  of  the  area  but  does  not  have  the  location  of  the 
control  points.  Our  subject  is  not  forced  to  follow  a  route,  but  is  given  an  en^ty  gun  with 
ammunition  being  at  the  controls.  He  is  also  required  to  punch  the  stanq)  at  each  control 
point  (to  confirm  he  visits  there).  If  he  shoots  the  hunter  the  hunter  must  return  to  the  start 
point  (or  so  he  is  told).  The  goal  is  to  reach  every  control  point  in  the  fastest  time,  while 
minimizing  the  number  of  times  you  are  shot. 

10.  Focus  More  on  Survey  Knowledge 

The  research  into  VEs  has  shown  it  to  be  most  effective  in  attaining  route  and 
landmark  knowledge  (see  Chapter  H).  This  is  not  to  say  that  survey  knowledge  cannot  be 
derived  from  a  VE,  just  that  it  is  not  transferred  as  readily  as  the  other  forms.  In  this 
thesis’  experiments,  survey  knowledge  was  not  tested  until  after  exposure  to  the 
environment,  which  may  have  produced  additional  learning  and  corrupted  the  results.  To 
more  accurately  test  the  ability  of  the  VE  to  transfer  survey  knowledge  we  must  test 
survey  knowledge  more  extensively  during  the  experiments.  One  avenue  to  explore  is  to 
force  the  subject  to  change  their  route  during  the  course  of  the  experiment.  By  informing 
the  subject  of  a  chemical  spill/forest  fire  and  making  them  adjust  their  route  without 
looking  at  a  map,  their  survey  knowledge  could  be  tested  earlier  than  the  unplanned  route 
test  in  the  current  model.  Goerger  suggested  that  another  way  of  doing  the  same  task 
would  be  to  reduce  the  number  of  points  but  upon  arrival  at  the  test  site  force  the  subjects 
to  run  their  route  backwards  [GOER  98b].  Either  method  would  be  more  preferable  to  the 
current  tests  which  were  not  done  by  all  subjects,  and  certainly  weren't  done  prior  to 
extensive  exposure  to  the  area  itself  for  an  extend  period  of  time  (at  least  30  minutes). 
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11.  Introduce  Competition  into  Training 

As  mentioned  above  in  Section  seven,  people  often  learn  best  when  they  do  not 
learn  alone.  In  this  case,  however,  they  will  not  be  learning  as  much  as  in  con:q)etition 
during  their  training.  Using  a  networked  virtual  environment,  two  subjects  would  be  run 
through  simultaneously.  Both  subjects  would  start  at  the  start  point  and  race  each  other 
for  the  controls.  Each  would  get  a  point  for  each  control  they  "captured"  first.  Each 
control  could  be  captured  only  once  and  must  be  captured  in  order.  This  would  put 
pressure  on  the  subjects  to  quickly  decide  upon  a  route  and  not  get  lost  en-route.  The 
game  could  be  run  several  times.  The  subjects  may  or  may  not  be  given  a  familiarization 
phase  with  the  area  first.  This  experiment  would  not  only  examine  the  effects  of 
conqjetition,  but  would  also  make  the  navigation  a  secondary  goal  of  the  training. 

12.  More  Guided  Instruction 

Another  lesson  in  the  learning  community  is  that  “Concrete  teaching  is  necessary 
before  students  attain  basic  knowledge  and  skills  to  make  free  exploration”[FENG  95]. 
Feng  notes  that  when  students  are  given  a  “sea  of  resources”  and  told  to  sink  or  swim, 
many  will  sink.  Without  experience  to  guide  them,  many  students  will  not  learn  efficiently 
as  they  do  not  know  how  to  learn.  With  VEs,  most,  if  not  all,  of  the  subjects  had  little 
experience  with  3D  models  and  environments.  With  a  map,  most  subjects  allowed  for 
minor  variances  between  what  the  map  showed  and  they  saw  in  the  real  world.  Some 
subjects  did  not  give  the  VE  the  same  leeway.  More  than  one  VE  subject  fully  expected  all 
the  trails  in  the  real  world  to  look  like  the  “yellow  brick  roads”  of  the  VE.  Other  subjects 
ignored  potential  reference  points  shown  in  the  VE  because  they  were  not  on  the  map  and 
were  uncertain  how  much  they  could  “trust”  the  VE.  Until  people  become  more  familiar 
with  VEs,  it  noay  be  necessary  to  guide  subjects  through  the  instruction  of  this  unfamiliar 
medium  more  than  the  sinq)le  15-minute  orientation  that  was  conducted  in  the 
e3q)eriments  so  far. 
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APPENDIX  A.  EXPERIMENT  OUTLINE 

1)  In  Brief^Consent  Foim 

a)  Time  -  5  Min 

b)  Location  -  CS  Student  Conf.  Room 

c)  OIC  -  CPT  Quay  B.  Jones 

d)  Materials  -  Consent  Form,  Privacy  Act  Statement,  Minimal  Risk  Consent  Form, 
Subject  Roster,  pencils,  Fort  Ord  Map  (confirm  the  subject  has  not  been  on  the 
course  terrain  before).  In  Briefing  Script 

2)  Color  Blindness  Test/Self  Evaluation  Questionnaires/Map  Reading  Test 

a)  Time  -  15  Min 

b)  Location  -  CS  Student  Conf  Room 

c)  OIC  -  CPT  Quay  B.  Jones 

d)  Materials  -  Color  Charts  (1  min).  Self  Ability  Evaluation  Sheet  (1  min),  Santa 
Barbara  Sense  of  Direction  Scale  Questionnaire  (3  min).  Map  Reading  Test  (5 
min),  pencil 

e)  Grading  (5  min) 

3)  Spatial  Orientation 

a)  Time  -  15  Min 

b)  Location  -  CS  Student  Conf  Room 

c)  OIC  -  CPT  Quay  B.  Jones 

d)  Materials  -  Guilford-Zimmerman  Aptitude  Tests  (10  min),  pencils,  answer  sheets, 

e)  Grading  and  Grouping  (5  min) 

f)  Groups 

i)  Group  A  -  Upper  50  percentile 

ii)  Group  B  -  Lower  50  percentile 

4)  Interface  Familiarization  (VE  Only) 

a)  Time  -  15  Min  minimum 

b)  Location  -  Graphics  Lab 

c)  OIC  -  CPT  Quay  B.  Jones 

d)  Materials  -SGI  machine.  Performer  Town  Model,  Flybox  instructions.  Virtual 
Environment  Briefing  Script,  Interface  Familiarization  Checklist 

e)  Movement  (15  min  minimum) 


107 


5)  Training 

a)  Map  Group  (Low  Fidelity  Map) 

(1)  Time  -  60  Min 

(2)  Location  -  CS  Student  Conf  Room 

(3)  QIC  -  CPT  Quay  B.  Jones 

(4)  Materials  -  1:24,000  Scale  Fort  Ord  Map,  Aerial  Photograph,  Participant 
Task  List,  Map  Marking  Instructions,  red  alcohol  marker,  alcohol  marker 
eraser,  pencil,  scratch  paper,  orienteering  clue  sheet.  Low  Fidelity  Map 
Group  Briefing  Script,  Training  Evaluation  Sheet 

b)  Map  Group  (Train-to-Standard) 

(1)  Time  -  Varies 

(2)  Location  -  CS  Student  Conf  Room 

(3)  QIC  -  CPT  Quay  B.  Jones 

(4)  Materials  -  Fort  Ord  Orienteering  Map,  Participant  Task  List,  Map 
Marking  Instructions,  red  alcohol  marker,  alcohol  marker  eraser,  pencil, 
scratch  paper,  orienteering  clue  sheet,  Train-to-Standard  Map  Group 
Briefing  Script,  Training  Evaluation  Sheet 

c)  Virtual  Environment  Group  (Low  Fidelity  Map) 

(1)  Time  -  60  Min 

(2)  Location  -  Graphics  Lab  CPT  Quay  B.  Jones 

(3)  OIC  -  CPT  Quay  B.  Jones 

(4)  Materials  -  Elvis  (SGI  )  w/  flybox  and  2rV40”  screen  configuration  or 
projector.  Fort  Ord  Model,  1:24,000  Scale  Fort  Ord  Map,  Aerial 
Photograph,  Participant  Task  List,  Map  Marking  Instructions,  Flybox 
instructions,  red  alcohol  marker,  alcohol  marker  eraser,  penciL  scratch 
paper,  orienteering  clue  sheet.  Low  Fidelity  Map  Virtual  Environment 
Briefing  Script,  Training  Evaluation  Sheet 

d)  Virtual  Environment  Group  (Train-to-Standard) 

(1)  Time  -  Varies 

(2)  Location  -  Graphics  Lab  CPT  Quay  B.  Jones 

(3)  OIC  -  CPT  Quay  B.  Jones 

(5)  Materials  -  Elvis  (SGI  )  w/  flybox  and  2rV40”  screen  configuration  or 
projector.  Fort  Ord  Model,  Fort  Ord  Orienteering  Map,  Participant  Task 
List,  Map  Marking  Instructions,  Flybox  instructions,  red  alcohol  marker, 
alcohol  marker  eraser,  pencil,  scratch  paper,  orienteering  clue  sheet,  Train- 
to-Standard,  Virtual  Environment  Briefing  Script,  Training  Evaluation 
Sheet 
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6)  Testing  (est  Time  120  Minutes  -  travel  to  Fort  Ord  Orienteering  Course,  run  the 
course,  and  return). 

a)  Time  -  Travel  Time  30  Min  (total);  Run  Course  90  Min;  Total  Time  (120  min) 

b)  Location  -  Fort  Ord  Orienteering  Course 

c)  OIC -CPT  (Juay B.  Jones 

d)  Materials  -  Clipboard  with  subject’s  map  &  designated  route,  conqjass,  Think  Out 
Loud  Instructions,  Data  Collection  Sheet,  red  pen  to  record  data,  blue  alcohol  pen, 
stop  watch/timer.  Color  Wheel  for  Tasks  3.1.  &  5.1,  White  Board  with  ten 
magnets,  rucksack  frame  w/GPS  system,  helmet  &  camera,  water,  first  aid  kit 
(cellular  phone).  Course  Briefing  Script,  blind  fold  (for  movement  to  course), 
spare  clue  sheet  &  color  wheel  arrows,  Tecnu  (for  poison  oak) 

e)  Tasks: 

i)  Task  1.  (Path  Knowledge)  Move  from  the  starting  point  to  Checkpoint  #1 

along  designated  route,  (measure  elapsed  time  and  number  of  errors;  mark 
deviation  from  route  on  map) 

ii)  Task  2.  (Path  Knowledge)  Move  from  Checkpoint  #1  to  Checkpoint  #2  along 

designated  route,  (measure  elapsed  time  and  number  of  errors;  mark 
deviation  from  route  on  map) 

iii)  Task  3.1.  (Survey  Knowledge)  Take  bearings  to  SP,  CP  #5,  and  CP  #9  at  the 

south  side  of  CP  #4) 

iv)  Task  3.2.  (Path  Knowledge)  Move  from  Checkpoint  #2  to  Checlq)oint  #3 

along  designated  route,  (measure  elapsed  time  and  number  of  errors;  mark 
deviation  from  route  on  map) 

v)  Task  4.  (Path  Knowledge)  Move  from  Checkpoint  #3  to  Checkpoint  #4  along 

designated  route,  (measure  elapsed  time  and  number  of  errors;  mark 
deviation  from  route  on  map) 

vi)  Task  5.1.  (Survey  Knowledge)  Take  bearings  to  CP  #1,  CP  #6,  and  CP  #8  at 

the  south  side  of  CP  #4) 

vu)Task  5.2.  (Path  Knowledge)  Move  from  Checlqpoint  #4  to  Checlqpoint  #5 
along  designated  route,  (measure  elapsed  time  and  number  of  errors;  mark 
deviation  from  route  on  noap) 

viii)  Task6.  (Path  Knowledge)  Move  from  Checkpoint  #5  to  Checkpoint  #6  along 

designated  route,  (measure  elapsed  time  and  number  of  errors;  mark 
deviation  from  route  on  map) 

ix)  Task  7.  (Path  Knowledge)  Move  from  Checlq)oint  #6  to  Checkpoint  #7  along 

designated  route,  (measure  elapsed  time  and  number  of  errors;  mark 
deviation  from  route  on  map) 

x)  Task  8.  (Path  Knowledge)  Move  from  Checkpoint  #7  to  Checkpoint  #8  along 

designated  route,  (measure  elapsed  time  and  number  of  errors;  mark 
deviation  from  route  on  map) 

xi)  Task  9.  (Path  Knowledge)  Move  from  Checlqpoint  #8  to  Checkpoint  #9  along 

designated  route,  (measure  elapsed  time  and  number  of  errors;  mark 
deviation  from  route  on  map) 
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xii)  Task  10.  (Survey  Knowledge)  Have  subject  indicate  bearing  and  route  he  must 

traverse  to  make  it  to  Checkpoint  #4.  Have  subject  return  to  Checkpoint 
#4.  Mark  route  and  any  turn  which  leads  the  subject  away  from  Checkpoint 
#4.  Allow  a  maximum  of  ten  minutes  to  return  to  Checkpoint  #4) 

xiii)  Task  11.  (Survey  Knowledge)  Have  subject  arrange  magnets  on  the  white 

board  indicating  the  location  of  the  starting  point  and  nine  checlqpoints. 
Measure  time  and  note  method  of  magnet  placement  (i.e.  in  order  of  visit, 
outside-in,  or  inside-out).  Take  picture  of  final  results  (allow  5  minutes 
maximum). 

g)  Error  (Definition) 

Subject  strays  from  designated  route  (5  meters  from  designated  route  on  a 
path/trail/road;  15  meters  from  cross  country  designated  route),  (record  one  error) 

7)  Debriefing. 

a)  Time  —  30  Min 

b)  Location  -  Graphics  Lab 

c)  OIC  —  CPT  Quay  B.  Jones 

d)  Materials  -  Clipboard  with  subject’s  map  &  designated  route.  Data  Collection 
Sheet,  red  pen  to  record  data,  GPS  system.  Troop  (PC)  w/  Arcview  and  Fort  Ord 
Maps,  digital  camera.  Participant  Questionnaire(s),  Researcher’s  Script 

e)  Administer  questionnaire(s);  down  load  GPS  datum  and  display  on  aerial  photo 
using  Arcview. 

f)  Discuss  route. 

i)  Have  the  subject  con^lete  the  Debriefing  Questionnaire.  Read  their  answers 

and  ask  for  any  clarification. 

ii)  Walk  the  subject  through  his  route  using  the  subjects  planned  route  and  the 

GPS  data  down  loaded  from  the  Message  Pad  and  plotted  on  the  aerial 
photo  in  Arcview. 

(a)  Have  the  subject  to  explain  why  they  deviated  from  their  route  at  those 
locations  where  the  two  differ. 

(b)  Have  the  subject  explain  when  &  how  they  determined  they  were  off 
course. 

(c)  Have  the  subject  explain  how  they  recovered. 

iii)  Ask  the  subject  if  he  would  have  done  anything  different  in  the  training  phase 

now  that  has  completed  the  experiment. 

iv)  How  much  time  does  the  subject  spend  playing  computer  games  or  working 

with  conputer  graphics  (more  than  an  hour  a  day,  a  couple  hours  a  week, 
once  or  twice  a  month,  rarely,  never)? 
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APPENDIX  B.  TASK  LISTING 


Task  1.  {Path  Knowledge)  Move  from  starting  point  to  Checkpoint  #1  along 
designated  route  (measure  elapsed  time  and  number  of  errors;  mark  deviation  from  route  on 
map). 

Task  2.  {Path  Knowledge)  Move  from  Checkpoint  #1  to  Checkpoint  #2  along 
designated  route  (measure  elapsed  time  and  number  of  errors;  mark  deviation  from  route  on 
map). 

Task  3.1.  {Survey  Knowledge)  Take  bearings  to  SP,  CP  #5,  and  CP  #9  at  the 
south  side  of  CP  #4). 

Task  3.2.  {Path  Knowledge)  Move  from  Checkpoint  #2  to  Checkpoint  #3  along 
designated  route  (measure  elapsed  time  and  number  of  errors;  mark  deviation  from  route  on 
map). 

Task  4.  {Path  Knowledge)  Move  from  Checkpoint  #3  to  Checkpoint  #4  along 
designated  route  (measure  elapsed  time  and  number  of  errors;  mark  deviation  from  route  on 
map). 

Task  5.1.  {Survey  Knowledge)  Take  bearings  to  CP  #1,  CP  #6,  and  CP  #8  at  the 
south  side  of  CP  #4). 

Task  5.2.  {Path  Knowledge)  Move  from  Checkpoint  #4  to  Checkpoint  #5  along 
designated  route  (measure  elapsed  time  and  #  errors;  mark  deviation  from  route  on  map). 

Task  6.  {Path  Knowledge)  Move  from  Checkpoint  #5  to  Checkpoint  #6  along 
designated  route  (measure  elapsed  time  and  number  of  errors;  mark  deviation  from  route  on 
map). 

Task  7.  {Path  Knowledge)  Move  from  Checkpoint  #6  to  Checkpoint  #7  along 
designated  route  (measure  elapsed  time  and  number  of  errors;  mark  deviation  from  route  on 
map). 

Task  8.  {Path  Knowledge)  Move  from  Checkpoint  #7  to  Checlqpoint  #8  along 
designated  route  (measure  elapsed  time  and  number  of  errors;  mark  deviation  from  route  on 
map). 
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Task  9.  {Path  Knowledge)  Move  from  Checlspoint  #8  to  Checkpoint  #9  along 
designated  route  (measure  elapsed  time  and  number  of  errors;  mark  deviation  from  route  on 
map). 

Task  Id.  {Survey  Knowledge)  Have  subject  indicate  bearing  and  route  he  must 
traverse  to  make  it  to  Checlq)oint  #4.  Have  subject  return  to  Checkpoint  #4  (mark  route 
and  any  turn  which  leads  the  subject  away  from  Checkpoint  #4.  Allow  a  maximum  of  ten 
minutes  to  return  to  Checkpoint  #4). 

Task  11.  {Survey  Knowledge)  Have  subject  arrange  magnets  on  the  white  board 
indicating  the  location  of  the  starting  point  and  nine  checkpoints.  Measure  time  and  note 
method  of  magnet  placement  (i.e.  in  order  of  visit,  outside  in,  or  inside  out).  Take  picture  of 
final  results  (allow  5  minutes  maximum). 
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APPENDIX  C.  BRIEFING  SCRIPTS 


1.  GENERAL 

The  scripts  in  the  appendix  <q)pear  in  the  same  fonnat  utilized  for  the  experiment  and  do 
not  follow  the  standard  thesis  format  utilized  in  the  chapters  of  this  document.  This  appendix 
consists  of  five  briefing  scripts:  In  Briefing,  Low  Fidelity  Map  Group  Briefing,  Train-To 
Standard  Map  Group  Briefing,  Low  Fidelity  VE  Briefing,  Train-to-Standard  VE  Briefing,  and 
the  Course  Briefing.  Each  participant  receives  the  In  Briefing  and  Course  Briefing.  The 
participants  are  exposed  to  either  the  Control  Group  Briefing,  Map  Group  Briefing,  or  Virtual 

t 

Environment  Briefing  depending  on  which  group  they  are  assigned.  This  appendix  also  contains 
the  Debriefing  hand  out. 

2.  IN  BRIEFING 

Welcome  to  the  Naval  Postgraduate  School’s  Conputer  Science  Department.  My  name 
is  CPT  Quay  B.  Jones.  Thank  you  for  your  assistance  with  today’s  experiment.  Today’s 
experiment  deals  with  dismounted  navigation  in  natural  terrain. 

This  experiment  is  not  a  test  of  your  intelhgence  or  performance.  Rather,  it  is  an 
evaluation  of  navigational  tools.  (For  Military  Personnel)  Your  petformance  will  not  be 
recorded  in  your  personnel  records  but  is  intended  for  research  purposes  only.  All  information 
collected  is  for  academic  research  only.  Prior  to  starting  the  experiment  you  will  be  asked  to 
read  and  sign  a  series  of  consent  forms.  Upon  signing  the  consent  forms,  you  will  take  self- 
evaluation,  map  reading,  and  spatial  orientation  exams.  After  the  tests,  you  will  under  go  a 
sixty-minute  train-up  period  prior  to  moving  to  the  navigation  course.  Upon  completing  the 
course,  you  will  be  brought  back  to  Spanagel  Hall  for  a  short  debriefing. 

If  there  are  no  questions,  please  read  and  sign  this  consent  form 
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3.  LOW  FTOEUTY  MAP  GROUP  BRIEFING 

In  front  of  you  is  a  1:24,000  scale  map  of  an  orienteering  course.  On  the  fl^  side 
of  the  map  is  the  same  map  enlarged  to  1:5,000  scale  but  with  no  additional  information. 
You  have  an  aerial  photograph  or  the  area.  You  also  have  a  clue  sheet  describing  the 
location  of  the  control  points  as  well  as  photos  of  the  control  points.  The  map  and  photos 
are  for  your  use  to  study  and  plan  the  route  you  will  be  using  to  navigate  the  course. 

You  have  sixty  minutes  to  study  the  map.  Your  planned  route  must  navigate  you 
through  the  nine  checkpoints  in  order.  (Show  the  participant  the  checfpoints  in  order 
then  point  out  each  checkpoint  in  the  photo.)  Beginning  at  the  designated  starting  point, 
you  will  go  to  CPI,  then  to  CP2,  then  to  CPS, ...  and  finally  to  CP9.  The  checkpoints  are 
described  in  the  clue  sheet  provided.  You  may  take  the  clue  sheet  with  you  when  you  go 
on  the  course.  Before  the  end  of  the  sixty-minute  study  phase,  you  will  mark  your  planned 
route  on  the  aerial  photograph  using  a  red  alcohol  marker. 

After  completing  the  study  phase,  you  will  be  taken  to  the  navigation  course  to  run 
the  route  you  designated  on  your  aerial  photograph.  While  navigating  the  course,  you  will 
not  have  the  map  nor  will  you  be  allowed  to  use  a  compass.  During  the  execution  of  the 
course,  you  may  request  a  one-minute  map  or  a  30-second  compass  check;  or  a  ninety- 
second  map  and  compass  check.  You  can  request  as  many  map  or  compass  checks  as  you 
wish,  but  each  check  will  be  recorded.  If  you  decide  to  deviate  from  your  previously 
planned  route,  you  may  request  the  map  to  mark  your  newly  planned  route. 

Do  you  have  any  questions  before  we  begin? 
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4.  TRAIN-TO-STANDARD  MAP  GROUP  BRIEFING 

In  front  of  you  is  a  map  of  an  orienteering  course.  You  also  have  a  clue  sheet 
describing  the  location  of  the  control  points  as  well  as  photos  and  screen  capture  images 
of  the  control  points.  The  map,  photos,  and  clue  sheet  are  for  your  use  to  study  and  plan 
the  route  you  will  be  using  to  navigate  the  course. 

You  will  now  study  the  map  and  photos  to  plan  and  memorize  your  route.  Your 
planned  route  must  navigate  you  through  the  nine  checkpoints  in  order.  (Show  the  subject 
the  checkpoints  in  order.)  Beginning  at  the  designated  starting  point,  you  will  go  to  CPI, 
then  to  CP2,  then  to  CPS, . . .  and  finally  to  CP9.  The  checkpoints  are  described  in  the  clue 
sheet  provided.  You  may  take  the  clue  sheet  with  you  when  you  go  on  the  course.  You 
will  mark  your  planned  route  on  the  map  using  a  red  alcohol  marker. 

Once  you  have  finished  planning  your  route,  you  should  study  it  in  order  to 
memorize  it.  When  you  feel  you  have  memorized  it,  (previous  subjects  have  taken  up  to  an 
hour  to  plan  and  study  their  route)  you  will  be  tested  prior  to  departing  for  the  course.  If 
you  do  not  adequately  pass  the  test,  you  will  again  be  given  time  to  study  your  route.  You 
must  pass  the  pre-course  test  to  standard  before  being  allowed  on  the  course. 

The  pre-course  test  will  consist  of  you  verbally  explaining  your  route  from 
memory  using  only  the  clue  sheet  provided.  You  will  explain  how  you  are  moving 
between  the  control  points,  and  what  clues  you  plan  to  use  to  assist  you.  During  this  test 
you  will  be  allowed  to  view  the  map  twice  in  order  to  refresh  your  memory  but  only  for  30 
seconds  each.  Be  as  detailed  and  descriptive  as  possible. 

After  completing  the  study  phase,  you  will  run  the  route  you  designated  on  your 
laminated  noap.  While  navigating  the  course,  you  will  not  have  the  map  nor  will  you  be 
allowed  to  use  a  con5)ass.  During  the  execution  of  the  course,  you  may  request  a  thirty 
seconds  map  or  con5)ass  check;  or  a  sixty-second  map  and  compass  check.  You  can 
request  as  many  map  or  compass  checks  as  you  wish,  but  each  check  will  be  recorded.  If 
you  decide  to  deviate  from  your  previously  planned  route,  you  may  request  the  map  to 
mark  your  newly  planned  route. 

Do  you  have  any  questions  before  we  begin? 
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5.  TRAIN-TO-STANDARD  VE  GROUP  BRIEFING 

Prior  to  beginning  the  study  phase  you  will  under  go  a  fifteen-minute  model 
familiarization  phase.  This  is  to  help  you  become  accounted  with  the  model  controls  prior 
to  starting  the  experiment.  The  model  you  will  be  using  for  this  phase  bares  no 
resemblance  to  the  actual  model  to  be  used  during  the  training  phase.  You  will  be  required 
to  show  proficiency  with  the  interface  prior  to  moving  on  to  the  terrain  model. 

In  fi-ont  of  you  is  the  3-screen  configuration,  a  joystick  interface,  and  a  list  of 
instruction  for  the  use  of  the  interface  (demo  controls).  Please  feel  free  to  explore  the 
environment  and  control  for  the  next  few  minutes.  When  you  feel  confident  with  the 
controls,  I  will  walk  you  through  a  serious  of  questions  to  demonstrate  your  expertise.  If 
you  feel  you  need  more  time  then  please  say  so. 

(Conduct  Familiarization  Phase;  after  demonstrating  their  proficiency  with  the 
interface,  load  up  the  terrain  model  and  begin  the  training  phase) 

In  firont  of  you  is  a  map  of  an  orienteering  course  as  well  as  a  high  fidelity  3-D 
model  of  the  terrain  depicted  on  the  map.  You  also  have  a  clue  sheet  describing  the 
location  of  the  control  points  as  well  as  photos  and  screen  capture  images  of  the  control 
points.  The  map,  photos,  and  VE  are  for  your  use  to  study  and  plan  the  route  you  will  be 
using  to  navigate  the  course. 

You  win  now  study  the  map  arid  VE  to  plan  and  memorize  your  route.  Your 
planned  route  must  navigate  you  through  the  nine  checkpoints  in  order.  (Show  the  subject 
the  checkpoints  in  order,  paging  through  each  page  as  you  go  to  each  checkpoint.) 
Beginning  at  the  designated  starting  point,  you  wiU  go  to  CPI,  then  to  CP2,  then  to  CP3, 
...  and  finaUy  to  CP9.  The  checkpoints  are  described  in  the  clue  sheet  provided.  You  may 
take  the  clue  sheet  with  you  when  you  go  on  the  course.  You  should  utilize  both  the  map 
and  VE  in  your  planning.  Once  you  have  decided  upon  a  route,  you  will  mark  your 
planned  route  on  the  map  using  a  red  alcohol  marker. 

Once  you  have  finished  planning  your  route,  you  should  use  the  map  and  VE  to 
assist  you  in  memorizing  it.  Once  you  feel  you  have  memorized  your  route  (previous 
subjects  have  taken  up  to  an  hour  or  more  to  plan  and  study  their  route)  you  will  tested 
prior  to  going  to  the  course.  If  you  do  not  pass  the  pre-course  test,  you  wOl  not  be  taken 
to  the  course,  but  will  instead  be  given  more  time  to  study  your  route.  You  must  pass  the 
pre-course  test  to  standard  before  being  allowed  on  the  real  course. 

The  pre-course  test  will  consist  of  you  being  placed  at  the  start  point  in  the  VE  and 
running  the  course  while  following  your  planned  route.  You  will  not  be  allowed  to  utilize 
the  15  meter  “pop-up”  view  during  this  test.  You  will  not  be  allowed  to  utilize  the  “you 
are  here”  map  during  this  test.  You  are  allowed  to  take  up  to  two  map  checks  using  the 
actual  laminated  map  of  the  area.  You  are  allowed  to  make  conq}ass  checks  as  needed,  but 
cannot  run  the  course  with  the  con^ass  “always  on”.  If  you  become  lost,  leave  your 
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planned  route  for  5  minutes,  leave  the  boundaries  or  encounter  a  check  point  out  of  order 
I  will  stop  you,  perform  a  mandatory  map  check,  record  a  mistake,  and  allow  you  to 
continue.  On  the  third  mistake  I  will  conclude  the  test  and  give  you  more  time  to  study. 
Note  that  conducting  a  search  for  a  nearby  point  does  not  count  as  being  lost. 

After  con^leting  the  pre-course  test,  you  will  be  taken  to  the  navigation  course  to 
run  the  route  you  designated  on  your  laminated  map.  While  navigating  the  course,  you  will 
not  have  the  map  nor  will  you  be  allowed  to  use  a  con:5)ass.  During  the  execution  of  the 
course,  you  may  request  a  thirty  seconds  map  or  con:q)ass  check;  or  a  sixty-second  map 
and  compass  check.  You  can  request  as  many  map  or  conq)ass  checks  as  you  wish,  but 
each  check  will  be  recorded.  If  you  decide  to  deviate  from  your  previously  planned  route, 
you  may  request  the  map  to  mark  your  newly  planned  route. 

Do  you  have  any  questions  before  we  begin? 
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6.  LOW  FTOELITY  MAP  VE  GROUP  BRIEFING 


Prior  to  beginning  the  study  phase  you  will  under  go  a  fifteen-minute  model 
familiarization  phase.  This  is  to  help  you  become  comfortable  with  the  model  controls 
prior  to  starting  the  experiment.  The  model  you  will  be  using  for  this  phase  bears  no 
resemblance  to  the  actual  model  to  be  used  during  the  training  phase.  You  will  be  required 
to  show  proficiency  with  the  interface  prior  to  moving  on  to  the  terrain  model. 

In  fi-ont  of  you  are  the  3-screen  configuration,  a  joystick  interface,  and  a  list  of 
instructions  for  the  use  of  the  interface  (demo  controls).  Please  feel  free  to  explore  the 
environment  and  controls  for  the  next  few  minutes.  When  you  feel  confident  with  the 
controls,  I  will  walk  you  through  a  serious  of  questions  to  demonstrate  your  expertise. 

(Conduct  Familiarization  Phase;  after  the  participant  demonstrates  proficiency 
with  the  interface^  load  up  the  terrain  model  and  begin  the  training  phase) 

In  front  of  you  is  a  1:24,000  scale  map  of  an  orienteering  course  as  well  as  a  high 
fidelity  3-D  model  of  the  terrain  depicted  on  the  map.  On  the  flip  side  of  the  map  is  the 
same  map  enlarged  to  1:5,000  scale.  No  additional  information  is  included  on  the  enlarged 
map.  Along  with  the  map,  you  have  an  aerial  photograph  of  the  region.  You  also  have  a 
clue  sheet  describing  the  location  of  the  control  points  ais  well  as  photos  and  screen 
capture  images  of  the  control  points.  The  maps,  photos,  and  VE  are  for  your  use  to  study 
and  plan  the  route  you  will  be  using  to  navigate  the  course. 

You  have  sixty  minutes  to  study  the  map  and  VE.  Your  planned  route  must 
navigate  you  through  the  nine  checkpoints  in  order.  (Show  the  participant  the 
checkpoints  in  order  then  point  out  each  checkpoint  in  the  photo.)  Beginning  at  the 
designated  starting  point,  you  will  go  to  CPI,  then  to  CP2,  then  to  CP3,  ...  and  finally  to 
CP9.  The  checkpoints  are  described  in  the  clue  sheet  provided.  You  may  take  the  clue 
sheet  with  you  when  you  go  on  the  course.  Before  the  end  of  the  sixty-minute  study 
phase,  you  will  mark  your  planned  route  on  the  aerial  photograph  using  a  red  alcohol 
marker. 

After  con^leting  the  training  phase,  you  will  be  taken  to  the  navigation  course  to 
run  the  route  you  designated  on  your  aerial  photograph.  While  navigating  the  course,  you 
will  not  have  the  map  nor  will  you  be  allowed  to  use  a  con^jass.  During  the  execution  of 
the  course,  you  may  request  a  one-minute  map  or  a  30-second  compass  check;  or  a  90- 
second  map  and  compass  check.  You  can  request  as  many  map  or  compass  checks  as  you 
wish,  but  each  check  will  be  recorded.  If  you  decide  to  deviate  from  your  previously 
planned  route,  you  may  request  the  map  to  mark  your  newly  planned  route. 

Do  you  have  any  questions  before  we  begin? 
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7.  COURSE  BRIEFING 


Pick-up  particq)ant  from  the  Graphics  Lab. 

Move  participant  to  the  Fort  Ord  orienteering  course. 

Move  participant  to  start  point: 

Brief  the  participant  on  animals  and  ammunition 

“You  are  at  the  start  point  of  the  Navigation  Course.  During  the  experiment,  I  may  stop 
you  and  ask  you  to  answer  questions.  You  must  navigate  the  nine  checkpoints  in  order. 
Each  control  point  will  be  identified  by  a  control  point  marker  (show  participant  a  control 
marker)  which  you  must  touch  prior  to  moving  to  the  next  control  point.  Once  you  touch 
a  control  marker,  I  will  tell  you  which  marker  it  is.  If  it  is  the  correct  marker,  I  will  give 
you  further  instructions.  If  it  is  the  incorrect  marker,  I  will  not  say  anything  other  then  the 
marker’s  number.  I  will  not  stop  you  unless  you  attempt  to  cross  the  course  boundaries 
(show  participant  the  boundaries).  You  may  request  the  compass  for  a  thirty  second 
con^ass  check;  the  map/maps  for  a  (30/60)  second  map  check;  or  the  map  and  conq)ass 
for  a  (60/90)  second  compass  and  map  check.  These  checks  will  be  recorded  and  timed  by 
me.  If  you  determine  that  you  would  like  to  change  your  route,  you  may  request  the  map 
and  a  blue  marker  to  mark  changes  to  your  proposed  route.  You  will  have  sixty  seconds 
to  mark  your  new  route.  You  may  request  an  additional  sixty  seconds  if  you  deem  it 
necessary.  You  have  sixty  minutes  to  make  it  as  far  as  you  can  along  your  planned  route. 
From  now  until  completion  of  the  navigation  course  do  not  interact  with  anyone.  Before 
you  begin,  do  you  have  any  questions?” 

TASK  1:  START  POINT  TO  CHECKPOINT  ONE. 

Task:  “Your  first  task  is  to  move  fi'om  the  start  point  to  checlqKjint  one  along  your 
designated  route.” 

Condition:  “Without  a  map  or  interaction  with  anyone  move  from  start  point  to 
checkpoint  one  along  your  preplanned  route.  If  you  deviate  from  the  designated  route  you 
will  be  allowed  to  continue  your  movement  unless  you  attenq)!  to  go  outside  the  course 
boundaries.  You  may  deviate  5m  from  your  route,  if  you  are  on  a  trail,  or  15m,  if  you  are 
conducting  cross-country  movement  before  you  are  assessed  an  error.  You  can  move  back 
and  forth  along  your  route  without  being  assessed  an  error.  If  you  deviate  fi'om  your  path 
for  more  then  15  continues  minutes  and  are  not  make  progress  towards  the  intended 
control  point,  I  will  stop  you,  show  you  your  location  on  the  map,  and  give  you  sixty 
seconds  to  mark  a  new  route  to  the  appropriate  control  point.” 

Standard:  “Do  the  best  you  can.” 

“Ready,...  Begin.” 
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TASK  2:  CHECKPOINT  ONE  TO  CHECKPOINT  TWO. 


Task:  “Checkpoint  one.  Your  next  task  is  to  move  from  the  checkpoint  one  to  checkpoint 
two  along  your  planned  route.  Conditions  and  standards  are  unchanged.” 

TASK  3.1.A,  B,  C:  SPATIAL  AWARENESS  TEST  1. 

Stop  timer 

Stop  participant  at  spatial  awareness  test  area. 

“Checkpoint  two.  Stop,  I  am  going  to  have  you  identify  the  direction  to  three 
checkpoints.” 

Place  the  color  wheel  platform  in  its  base  on  the  south  side  of  checkpoint. 

Task:  “Identify  the  direction  to  the  start  point,  checkpoint  five,  and  checkpoint  nine.” 
Show  participant  arrows  as  you  state  their  names. 

Condition:  “Given  a  color  coded,  360-degree  wheel  and  three  arrows,  identify  the 
direction  to  the  start  point,  checkpoint  five,  and  checkpoint  nine  by  placing  the  appropriate 
arrow  in  the  direction  of  its  checkpoint.” 

Standard:  “Unchanged.” 

Record  the  time  it  takes  the  participant  to  perform  the  Wheel  task  and  the 
orientation  of  the  participant  (looking  north,  south,  east,  rotates  in  the  direction  of  the 
arrows,  etc).  Once  done,  photo  graph  the  wheel,  remove  wheel  platform  from  its  stand, 
and  have  participant  continue  to  checkpoint  three. 

TASK  3.2:  CHECKPOINT  TWO  TO  CHECKPOINT  THREE. 

Task:  “Your  next  task  is  to  move  from  the  checkpoint  two  to  checkpoint  three  along  your 
planned  route.  Conditions  and  standards  are  unchanged.  Ready,...  Begin.” 

Start  timer 

TASK  4:  CHECKPOINT  FOUR  TO  CHECKPOINT  FIVE. 

Task:  “Checkpoint  three.  Your  next  task  is  to  move  from  the  checkpoint  three  to 
checkpoint  four  along  your  planned  route.  Conditions  and  standards  are  unchanged.” 
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TASK  5.1.A,  B,  C:  SPATIAL  AWARENESS  TEST  I. 

Stop  timer 

Stop  participant  at  spatial  awareness  test  area. 

“Checlq)omt  four.  Stop,  I  am  going  to  have  you  identify  the  direction  to  three 
checkpoints.” 

Place  the  color  wheel  platform  in  its  base  on  the  south  side  of  checkpoint 
Task:  “Identify  the  direction  to  checkpoint  one,  checkpoint  six,  and  checkpoint  eight.” 
Show  participant  arrows  as  you  state  their  names. 

Condition:  “Given  a  color  coded,  360-degree  wheel  and  three  arrows,  identify  the 
direction  to  checkpoints  one,  six,  and  eight  by  placing  the  appropriate  arrow  in  the 
direction  of  its  checlq)oint.” 

Standard:  “Unchanged.” 

Record  the  time  it  takes  the  participant  to  perform  the  Wheel  task  and  the  orientation 
of  the  participant  (looking  north,  south,  east,  rotates  in  the  direction  of  the  arrows, 
etc).  Once  done,  photo  graph  the  wheel,  remove  wheel  platform  from  iti  stand,  and 
have  participant  continue  to  checkpoint  five. 

TASK  5.2;  CHECKPOINT  TWO  TO  CHECKPOINT  THREE. 

Task:  “Your  next  task  is  to  move  from  the  checkpoint  four  to  checkpoint  five  along  your 
planned  route.  Conditions  and  standards  are  unchanged.  Ready,...  Begiu.” 

Start  timer 


TASK  6:  CHECKPOINT  FIVE  TO  CHECKPOINT  SIX. 

Task:  “Checlqpoint  five.  Your  next  task  is  to  move  from  the  checkpoint  five  to 
checkpoint  six  along  your  planned  route.  Conditions  and  standards  are  unchanged.” 

TASK  7;  CHECKPOINT  SIX  TO  CHECKPOINT  SEVEN. 

Task:  “Checkpoint  six.  Your  next  task  is  to  move  from  the  checlqpoint  six  to  checkpoint 
seven  along  your  planned  route.  Conditions  and  standards  are  unchanged.” 
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TASK  8:  CHECKPOINT  SEVEN  TO  CHECKPOINT  EIGHT. 


Task:  “Checkpoint  seven.  Your  next  task  is  to  move  from  the  checkpoint  seven  to 
checkpoint  eight  along  your  planned  route.  Conditions  and  standards  are  unchanged.” 

TASK  9:  CHECKPOINT  EIGHT  TO  CHECKPOINT  NINE. 

Task;  “Checkpoint  eight.  Your  next  task  is  to  move  from  the  checkpoint  eight  to 
checkpoint  nine  along  your  planned  route.  Conditions  and  standards  are  unchanged.” 

TASK  10.1:  CHECKPOINT  4  IDENTIFICATION. 

While  standing  at  checlqpoint  nine: 

Stop  timer 

Task:  “Checkpoint  nine,  finish  point.  Your  next  task  is  to  identify  the  location  of 
checkpoint  four  from  where  you  are.” 

Condition:  “Point  to  checkpoint  four  and  tell  me  where  checkpoint  four  is  from  here. 
(i.e.,  twenty  meters  and  in  this  direction).” 

Standard:  “Unchanged.” 

TASK  10.2:  DESCRIBE  ROUTE  FROM  CHECKPOINT  NINE  TO  START 
POINT 

Task:  “Your  next  task  is  to  describe  what  you  consider  the  easiest  route  you  would  take 
to  move  from  here  to  checkpoint  four.” 

Condition:  “Without  a  map,  describe  the  route  you  would  take  to  move  from  checkpoint 
nine  to  chec]q)oint  four.” 

Standard:  “Unchanged.” 

TASK  10.3:  CHECKPOINT  NINE  TO  START  POINT  (if  described  route  would 
take  them  in  the  general  location  of  the  start  point) 

Task:  “Your  next  task  is  to  move  from  checkpoint  nine  to  checkpoint  four  using  the  route 
you  just  described.” 

Condition:  “Again,  do  not  interact  with  anyone  to  include  the  researcher.  You  may  not 
request  a  map  or  a  conpass  check.” 

Standard:  “You  have  ten  minutes,  otherwise  standards  are  unchanged.” 
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“Re^y,...  Begin” 

Start  timer 

Reach  checkpoint  #4  or  ten  minutes  has  elapsed. 

FINISH 

Stop  timer 

“Stop.  Congratulations  you  have  completed  the  navigation  portion  of  this  experiment. 
We  will  now  return  to  the  vehicle  for  one  final  test  before  returning  to  the  laboratory.” 

TASK  11:  WHITE  BOARD  TEST. 

Task:  “Your  final  task  is  to  create  a  top  down  representation  of  the  start  point  and  nine 
control  points.” 

Condition:  “Without  a  map  or  interaction  with  anyone  take  the  ten  magnets  labeled  with 
the  start  point  and  nine  checkpoints  (show  the  participant  the  magnets)  and  place  them 
on  a  clean  white  board  in  proper  perspective  to  each  other.  You  are  attenq)ting  to  create 
a  top  down  view  of  the  checkpoints,  actual  distance  between  points  does  not  matter, 
however,  relative  locations  to  each  checkpoint  does.  Until  you  feel  you  are  finished  or 
five  nainutes  has  elapsed,  you  may  place  and  move  the  magnets  as  you  wish. 

Standard:  “Do  the  best  you  can.” 

“Any  questions, . . .  Ready, . . .  Begin.” 

Start  timer 

Stop  (he  timer  when  the  participant  indicates  he  has  finished  or  ten  minutes  has 
elapsed,  which  ever  occurs  first.  Observe  the  participant  and  note  his  method  for 
piling  the  magnets  (Le.  in  order  of  visit,  outside  in,  or  inside  out).  Take  a  picture  of 
the  final  results  (allow  participant  5  minutes  maximum  to  perform  the  task). 


Stop  timer 

“Stop.  Congratulations  on  completing  the  final  task  for  this  experiment.  We  will  now 
return  to  NFS  for  a  final  debriefing  session.” 

Move  participant  back  to  the  Graphics  Lab  for  debriefing. 
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8.  DEBRIEFING 


The  use  of  virtual  environments  in  training  and  education  has  been  an  expanding 
field  for  the  last  two  decades.  With  recent  developments  in  con:q)uter  systems,  virtual 
reality  models  are  now  able  to  display  much  higher  fidelity.  In  order  to  insure  we  are 
providing  a  positive  training  transfer  and  properly  replicating  real  world  environments, 
research  is  being  conducted  in  the  levels  of  detail  required  in  models. 

The  study  you  have  just  conpleted  is  concerned  with  gathering  information  on 
how  individuals  navigate  through  complex  virtual  environments.  You  spent  a  session 
planning  and  stud3dng  a  route  demonstrating  route  knowledge  Finally,  you  demonstrated 
spatial  knowledge  of  the  terrain  through  estimating  bearings  to  known  points  and 
movement  to  an  unplanned  location. 

Four  separate  groups  were  examined  in  order  to  determine  performance  levels.  All 
four  groups  were  given  a  map  on  which  they  designated  their  routes  prior  to  running  the 
navigation  course.  The  first  group  was  only  allowed  to  study  a  1:24,000  map  for  60 
minutes.  The  second  group  was  given  the  1:24,000  map  and  allowed  to  maneuver  through 
a  real  time,  high  fidelity  virtual  representation  of  the  terrain  for  60  minutes.  The  third 
group  had  only  a  1:5,000  scale  map  but  had  to  train  to  a  set  standard  before  being  allowed 
on  the  actual  outdoor  course.  The  fourth  group  had  the  1:5,000  scale  map  and  the  VE  and 
also  had  to  complete  the  training  to  a  standard  before  running  the  course. 

The  research  personnel  observed  and  recorded  information  based  on  the 
experience  and  behavior  of  the  participants  in  order  to  gather  the  information  equipped  for 
the  redesign  and  implementation  of  a  more  useful  virtual  model.  The  notes  and 
observations  collected  will  be  used  for  the  purpose  of  establishing  standards  for  model 
development. 

Your  assistance  in  this  project  will  contribute  to  the  production  of  more  useful 
virtual  environments  that  provide  users  with  spatial  knowledge  and  better  navigational 
skills.  With  the  information  gathered  from  your  experience  and  the  experience  of  other 
partic^ants,  we  are  discovering  what  people  generally  use  as  navigational  cues  in  the 
virtual  and  real  world  environments.  This  information  will  assist  in  the  design  of  future 
virtual  reality  models  that  will  be  adaptive  to  a  variety  of  individual  needs. 

If  you  have  any  questions  about  the  study,  please  ask  your  research  assistant. 
Until  30  August  1999,  please  do  not  discuss  this  experiment  with  anyone  except  our 
research  personnel  to  prevent  influencine  any  future  participants.  Thank  you  for  your 
participation  in  this  study. 

The  research  supervisor,  CPT  Quay  B.  Jones,  for  this  study  can  be  contacted  at 
(408)  656  —  4077  or  Email:  jones@cs.nps.navy.mil. 
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APPENDIX  D.  CONSENT  FORMS 


1.  GENERAL 

The  forms  in  the  appendix  appear  in  the  same  format  utilized  for  the  experiment  and  do 
not  follow  the  standard  thesis  format  utilized  in  the  chapters  of  this  document.  This  appendix 
consists  of  three  documents:  Consent  Form,  Minimal  Risk  Consent  Statement,  and  the  Privacy 
Act  Statement.  Each  participant  is  required  to  read  and  sign  these  documents  before  he  is 
allowed  to  participate  in  the  study.  A  research  monitor  observes  and  verifies  the  signing  of 
each  document.  The  format  and  content  of  these  documents  is  based  on  the  forms  used  in 
MAJ  William  Banker’s  land  navigation  experiment  [BANK  97]  and  MAJ  Simon  R.  Goerger’s 
land  navigation  experiment  [GOER  98b]. 
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2.  CONSENT  FORM 


PARTICIPANT  CONSENT  FORM 

1.  Introduction.  You  are  invited  to  participate  in  a  study  of  spatial  awareness  of  natural 
and  virtual  environments.  With  information  gathered  from  you  and  other  participants, 
we  hope  to  discover  insight  on  navigational  aids  used  to  move  through  virtual 
environments  during  dismounted  navigation  of  natural  terrain.  We  ask  you  to  read  and 
sign  this  form  indicating  that  you  agree  to  be  in  the  study.  Please  ask  any  questions  you 
may  have  before  signing. 

2.  Background  Information.  The  Naval  Postgraduate  School  NPSNET  Research  Group 
is  conducting  this  study. 

3.  Procedures.  If  you  agree  to  participate  in  this  study,  the  researcher  will  explain  the 
tasks  in  detail.  There  will  be  two  sessions:  a)  30  pretest  phase  and  2)  training  and 
execution  phases  lasting  approximately  five  hours  in  duration,  during  which  you  will  be 
expected  to  acconplish  a  number  of  tasks  related  to  navigating  natural  terrain. 

4.  Risks  and  Benefits.  This  research  involves  no  risks  or  discomforts  greater  then  those 
encountered  in  ordinary  hike  through  rolling,  wooded  terrain.  The  benefits  to  the 
participants  are  gaining  techniques  for  enhancing  spatial  knowledge  of  unfamiliar 
environments  and  contributing  to  current  research  in  human-computer  interaction. 

5.  Compensation.  No  tangible  reward  will  be  given.  A  copy  of  the  results  will  be 
available  to  you  at  the  conclusion  of  the  experiment. 

6.  Confidentiality.  The  records  of  this  study  will  be  kept  confidential.  No  information 
will  be  publicly  accessible  which  will  possibly  identify  you  as  a  participant. 

7.  Voluntary  Nature  of  the  Study.  If  you  agree  to  participate,  you  are  free  to  withdraw 
from  the  study  at  any  time  without  prejudice.  You  wiU  be  provided  a  copy  of  this  form 
for  your  records. 

8.  Points  of  Contact.  If  you  have  any  further  questions  or  comments  after  the  complotion 
of  the  study,  you  may  contact  the  research  supervisor,  CPT  Quay  B.  Jones,  at  (408) 
656  -  4077  (Email:  jones@cs.nps.navy.mil). 

9.  Statement  of  Consent.  I  have  read  the  above  information.  I  have  asked  aU  question 
and  have  had  my  questions  answered.  I  agree  to  participate  in  this  study. 


Participant’s  Signature 


Date 


Researcher’s  Signature 


Date 
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3.  MINIMAL  RISK  CONSENT  STATEMENT 


NAVAL  POSTGRADUATE  SCHOOL,  MONTEREY,  CA  93943 
MINIMAL  RISK  CONSENT  STATEMENT 

Participant:  VOLUNTARY  CONSENT  TO  BE  A  RESEARCH  PARTICIPANT  IN:  Virtual 

Environments  and  Navigation  in  Natural  Environments 

1.  I  have  read,  understand  and  been  provided  "Information  for  Participants"  that  provides  the 
details  of  the  below  acknowledgments. 

2.  I  understand  that  this  project  involves  research.  An  explanation  of  the  purposes  of  the 
research,  a  description  of  procedures  to  be  used,  identification  of  experimental  procedures, 
and  the  extended  duration  of  my  participation  have  been  provided  to  me. 

3.  I  imderstand  that  this  project  does  not  involve  more  than  minimal  risk.  I  have  been  informed 
of  any  reasonably  foreseeable  risks  or  discomforts  to  me. 

4.  I  have  been  informed  of  any  benefits  to  me  or  to  others  that  may  reasonably  be  expected  from 
the  research. 

5.  I  have  signed  a  statement  describing  the  extent  to  which  confidentiality  of  records  identifying 
me  will  be  maintained. 

6.  I  have  been  informed  of  any  con:q)ensation  and/or  medical  treatments  available  if  injury 
occurs  and  is  so,  what  they  consist  of,  or  where  further  infornaation  may  be  obtained. 

7.  I  understand  that  my  participation  in  this  project  is  voluntary,  refusal  to  participate  will 
involve  no  penalty  or  loss  of  benefits  to  which  I  am  otherwise  entitled.  I  also  understand  that 
I  may  discontinue  participation  at  any  time  without  penalty  or  loss  of  benefits  to  which  I  am 
otherwise  entitled. 

8.  I  understand  that  the  individual  to  contact  should  I  need  answers  to  pertinent  questions  about 
the  research  is  Rudy  Darken,  Ph.D.,  Principal  Investigator,  and  about  my  rights  as  a  research 
participant  or  concerning  a  research  related  injury  is  the  Modeling  Virtual  Environments  and 
Simulations  Chairman.  A  full  and  responsive  discussion  of  the  elements  of  this  project  and 
my  consent  has  taken  place. 

Medical  Monitor:  Flight  Surgeon,  Naval  Postgraduate  School 


Signature  of  Principal  Investigator 

Date 

Signature  of  Volunteer 

Date 

Signature  of  Witness 

Date 
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4.  PRIVACY  ACT  STATMENT 


NAVAL  POSTGRADUATE  SCHOOL,  MONTEREY,  CA  93943 
PRIVACY  ACT  STATEMENT 

1.  Authority:  Naval  Instruction 

2.  Purpose:  Spatial  Cognition  information  will  be  collected  to  enhance  knowledge,  or 
to  develop  tests,  procedures,  and  equipment  to  improve  the  development  of  Virtual 
Environments. 

3.  Use:  Spatial  Cognition  information  will  be  used  for  statistical  analysis  by  the 
Departments  of  the  Navy  and  Defense,  and  other  U.S.  Government  agencies, 
provided  this  use  is  con^atible  with  the  purpose  for  which  the  information  was 
collected.  Use  of  the  information  may  be  granted  to  legitimate  non-govemment 
agencies  or  individuals  by  the  Naval  Postgraduate  School  in  accordance  with  the 
provisions  of  the  Freedom  of  Information  Act. 

4.  Disclosure/Confidentiality: 

a.  I  have  been  assured  that  my  privacy  will  be  safeguarded.  I  will  be  assigned  a 
control  or  code  number  which  thereafter  will  be  the  only  identifying  entry  on 
any  of  the  research  records.  The  Principal  Investigator  will  maintain  the  cross- 
reference  between  name  and  control  number.  It  will  be  decoded  only  when 
beneficial  to  me  or  if  some  circumstances,  which  is  not  apparent  at  this  time, 
would  make  it  clear  that  decoding  would  enhance  the  vidue  of  the  research  data. 
In  all  cases,  the  provisions  of  the  Privacy  Act  Statement  will  be  honored. 

b.  I  understand  that  a  record  of  the  information  contained  in  this  Consent  Statement 
or  derived  fi-om  the  experiment  described  herein  will  be  retained  permanently  at 
the  Naval  Postgraduate  School  or  by  higher  authority.  I  voluntarily  agree  to  its 
disclosure  to  agencies  or  individuals  indicated  in  paragraph  3  and  I  have  been 
informed  that  failure  to  agree  to  such  disclosure  may  negate  the  purpose  for 
which  the  experiment  was  conducted. 

c.  I  also  understand  that  disclosure  of  the  requested  information,  including  my 
Social  Security  Number,  is  voluntary. 


Signature  of  Volunteer  Name,  Grade/Rank  (if  applicable)  DOB  SSN  Date 


Signature  of  Witness  Date 
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APPENDIX  E.  QUESTIONNAIRES  AND  TESTS 

1.  GENERAL 

The  items  in  the  appendix  appear  in  the  same  format  utilized  for  the  experiment 
and  thus  do  not  follow  the  standard  thesis  format  utilized  in  the  chapters  of  this 
document.  This  appendix  consists  of  eight  documents:  Land  Navigation  Questionnaire, 
Self  Ability  Evaluation,  Santa  Barbara  Sense-of-Dkection  Scale,  Map  Reading  Test, 
Guilford-Zimmerman  Aptitude  Survey  (cover  only).  Practice  Model  Test,  and  two 
Debriefing  Questionnaires.  This  tests  and  questionnaires  are  the  same  as  in  Goerger’s 
thesis  [GOER  98b]. 

The  Land  Navigation  Questionnaire  (Appendix  E.2)  gathers  general  navigational 
background  of  the  participant.  The  participant  con(q)letes  this  prior  to  beginning  the 
experiment. 

The  Self  Ability  Evaluation  (Appendix  E.3)  is  a  qualitative  self  analysis  of  an 
individual’s  navigational  ability.  It  provides  a  participant  with  general  limits  from  which 
to  appraise  his  perceived  navigation  aptitude.  The  left  end  of  the  scale  is  valued  at  0.00 
and  the  right  end  of  the  bar  line  is  valued  at  1.00.  Values  measured  from  0.00  to  0.33  are 
assessed  as  beginning  navigators.  From  0.33  to  0.66  is  ranked  as  an  intermediate 
navigator.  Values  of  0.66  to  1.00  are  evaluated  as  experts.  This  test  was  included  to 
negate  any  bias  inherent  in  the  word  “beginner”  from  the  Land  Navigation  Questionnaire. 

The  Santa  Barbara  Sense-of-Direction  Scale  (Appendix  E.4)  is  a  quantitative  self- 
evaluation  of  navigational  ability.  The  University  of  Cahfomia  at  Santa  Barbara 
developed  the  scale.  An  individual’s  score  is  calculated  by  reversing  the  values  of 
questions  2,  6,  8,  10,  11,  12,  13,  and  15.  For  exan^le,  if  the  participant  answered 
question  number  two  as  “3”,  the  question  is  given  a  numerical  value  of  “5”.  Once  the 
values  for  the  above  questions  are  reversed,  sum  the  value  of  each  question  and  divide  the 
total  by  the  number  of  questions  answered.  The  lower  the  resulting  score  the  more 
confidant  an  individual  is  in  their  navigational  abilities.  The  test  scale  has  a  mean  score  of 
3.54  with  a  standard  deviation  of  1.03.  For  this  ejq)eriment,  most  individuals  expressed 
high  confidence  in  their  navigational  abilities. 

The  Map  Reading  Test  (Appendix  E.5)  consists  of  twenty  questions  dealing  with 
terrain  identification.  The  test  is  designed  to  determine  an  individual’s  proficiency  in 
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reading  the  terrain  features  on  a  map  and  associating  them  to  teal  world  terrain  features. 
The  first  fifteen  questions  concern  properly  identifying  terrain  features  from  1:50,000 
scale  military  maps.  The  final  five  questions  dealt  with  matching  images  of  terrain 
features  to  map  depictions  of  terrain  features.  The  answers  for  the  test  are  listed  in  Table 
E.l.  Each  question  is  worth  one  point.  If  a  participant  misidentifies  a  linear  terrain  feature 
they  receive  0.5  points  for  the  question.  For  exan5)le,  if  the  terrain  feature  is  a  stream  and 
the  participant  classifies  it  as  a  road,  they  receive  0.5  points  for  the  question.  However,  if 
the  participant  describes  a  stream  as  a  draw,  they  receive  no  credit.  Participants  must 
score  65%  (13  out  of  20)  or  better  to  be  allowed  to  participate  in  the  study. 


Question 

Answer 

Feature 

1.1 

B 

Draw 

1.2 

I 

Spur/Finger 

1.3 

H 

Saddle 

1.4 

A 

Depression 

1.5 

C 

Hill  Top 

2.1 

F 

Road/Trail 

2.2 

B 

Draw 

2.3 

E 

Ridge  Line 

2.4 

L 

Valley 

2.5 

I 

Spur/Finger 

3.1 

I 

Spur/Finger 

3.2 

C 

Hill  Top 

3.3 

G 

RoadyTrail  Intersection 

3.4 

J 

Stream/River 

3.5 

B 

Draw 

4.1 

F 

Road/Trail 

4.2 

D 

Hill  Top 

4.3 

A 

Road 

4.4 

E 

Saddle 

4.5 

C 

Spur/FingCT 

Table  E.l  Map  Test  Answer  Key 
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The  Guilford-Zimmerman  Aptitude  Survey  (Appendix  E.6)  evaluates  an 
individual’s  spatial  orientation  ability.  The  results  of  this  test  are  conq)ared  to  a  pool  of 
national  test  scores  to  determine  if  a  participant  is  above  or  below  the  national  average 
for  spatial  orientation.  Those  above  the  national  average  are  describe  as  having  a  “high” 
GZ  score,  or  high  spatial  ability.  Those  scoring  below  the  average  are  classified  as  “low”. 
The  experiment  groups  were  set  up  to  evenly  distribute  high  and  low  ability  subjects 
evenly  between  the  groups. 

The  Practice  Model  Test  (Appendix  E.7)  is  administered  to  each  VE  participant 
prior  to  moving  onto  the  actual  course  model.  It  is  used  to  provide  the  participant  with 
familiarity  with  the  interface  functions.  Each  virtual  environment  participant  was 
required  to  con^lete  each  task  of  the  Practice  Model  Test.  After  con5)leting  the  test,  a 
participant  is  retested  on  any  functions  they  failed  to  properly  employ  until  he  is  able  to 
do  so. 

The  Debriefing  Questioimaires  (Appendices  E.8  and  E.9)  are  administered  upon 
return  from  running  the  course  and  prior  to  the  final  review  of  the  participant’s  route. 
Participants  in  the  Map  Only  Groups  received  the  questionnaire  in  Appendix  E.8.  Virtual 
Environment  participants  receive  the  questionnaire  in  Appendix  E.9  that  has  an  additional 
page  containing  questions  related  to  the  VE  and  its  interface.  The  questions  are  designed 
to  provide  a  qualitative  analysis  of  the  training  materials  and  course  [GOER  98b].  A  five 
point  scale  (1-5)  is  used  for  the  questionnaire.  The  last  page  of  the  questionnaire  is  used 
to  determine  what  details  subjects  think  are  necessary  for  inclusion  in  a  useful  VE. 
Goerger  deliberately  left  stream/rivers  off  of  the  list  to  see  how  much  attention  subjects 
were  paying. 

The  raw  scores  from  these  tests  and  questionnaires  are  listed  in  Appendix  O. 
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2.  LAND  NAVIGATION  QUESTIONNAIRE 


Name: _  Age: _  Sex:. 

Branch  of  Service: _  Rank: _ 


1)  Where  did  you  first  learn  to  navigate? 

a)  Scouting,  Boys/Girls  Club 

b)  Parents 

c)  Friend 

d)  ROTC/Academy 

e)  Basic  Training 

f)  Officer  Candidate  School 

g)  Officers  Basic  Course 

h)  Other: _ 

2)  How  many  years  have  you  been  Orienteering/Navigating? 

a)  less  then  a  year 

b)  one  year  or  more 

c)  two  years  or  more 

d)  five  years  or  more 

e)  ten  years  or  more 

3)  At  what  level  would  you  classify  your  navigating  abilities? 

a)  Novice/Beginner 

b)  Intermediate/ Average 

c)  Expert/Advanced 

4)  How  many  Land  navigation  or  Orienteering  courses  have  you  done  in  the  last  year? 


5)  The  land  navigation  course  runs  through  varying  degrees  of  vegetation  and  over 
rolling  terrain.  It  will  require  you  to  negotiate  a  Stance  of  no  more  than  three  miles  in 
one  hour.  Do  you  have  any  physical  disabilities  that  would  prevent  you  from  executing 
this  task?  Yes/No 
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3.  SELF  ABILITY  EVALUATION 


Participant  ID; 


The  following  bar  line  depicts  the  navigation  ability  evaluation  of  an  average 
infantry  officer  with  five  years  experience.  The  “X”  indicates  his  ability  level 


Knows  how  to  Navigates  with 

read  a  map  no  errors; 

Rarely  looks 
at  map 


Place  an  “X”  on  the  line  below  were  you  feel  your  navigational  abilities  are  at  this 

time. 


Knows  how  to  Navigates  with 

read  a  map  no  errors; 

Rarely  looks 
at  map 
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4.  SANTA  BARBARA  SENSE-OF-DIRECTION  SCALE 


Participant  ID: _  Date: _  SEX:  F  M  AGE: _ 

This  questionnaire  consists  of  several  statements  about  your  spatial  and 
navigational  abilities,  preferences,  and  experience.  After  each  statement,  you  should 
circle  a  number  to  indicate  your  level  of  agreement  with  the  statement.  Circle  “1”  if  you 
strongly  agree  that  the  statement  applies  to  you,  ‘7”  if  you  strongly  disagree,  or  some 
number  in  between  if  your  agreement  is  intermediate.  Circle  “4”  if  you  neither  agree  nor 
disagree. 

1 .  lam  very  good  at  directions. 

strongly  agree  1  2  3  4  5  6  7  strongly  disagree 

2.  I  have  a  poor  memory  for  were  I  left  things, 

strongly  agree  1  2  3  4  5  6  7  strongly  disagree 

3.  lam  very  good  at  judging  distances. 

strongly  agree  1  2  3  4  5  6  7  strongly  disagree 

4.  My  “sense  of  direction”  is  very  good. 

strongly  agree  1  2  3  4  5  6  7  strongly  disagree 

5.  I  tend  to  think  of  my  environment  in  terms  of  cardinal  directions  (N,  S,  E,  W) 

strongly  agree  1  2  3  4  5  6  7  strongly  disagree 

6.  I  very  easily  get  lost  in  a  new  city. 

strongly  agree  1  2  3  4  5  6  7  strongly  disagree 

7.  I  enjoy  reading  maps. 

strongly  agree  1  2  3  4  5  6  7  strongly  disagree 

8.  I  have  trouble  understanding  directions. 

strongly  agree  1  2  3  4  5  6  7  strongly  disagree 

(turn  over  and  continue) 
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9.  lam  very  good  at  reading  maps. 

strongly  agree  1  2  3  4  5  6  7  strongly  disagree 

10. 1  don’t  remember  routes  very  well  while  riding  as  a  passenger  in  a  car. 

strongly  agree  1  2  3  4  5  6  7  strongly  disagree 

1 1 . 1  don’t  enjoy  giving  directions. 

strongly  agree  1  2  3  4  5  6  7  strongly  disagree 

12.  It’s  not  important  to  me  to  know  where  I  am 

strongly  agree  1  2  3  4  5  6  7  strongly  disagree 

13. 1  usually  let  someone  else  do  the  navigational  planning  for  long  trips, 

strongly  agree  1  2  3  4  5  6  7  strongly  disagree 

14. 1  can  usually  remember  a  new  route  after  I  have  traveled  it  only  once, 

strongly  agree  1  2  3  4  5  6  7  strongly  disagree 

15. 1  don’t  have  a  very  good  “mental  map”  of  my  environment, 

strongly  agree  1  2  3  4  5  6  7  strongly  disagree 
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5.  MAP  READING  TEST 


The  following  is  a  list  of  terrain  features  commonly  found  on  military  and/or 
orienteering  maps.  Using  the  list  of  terrain  features,  identify  the  most  predominate 
terrain  feature  within  each  circle  and  place  your  answer  in  the  space  provided.  Each 
terrain  feature  from  the  list  may  be  used  more  than  once  or  not  at  all. 


A.  Depression 

B.  Draw 

C.  HiUTop 

D.  Lake/Pond 

E.  Ridge  Line 

F.  Road/Trail 

G.  Road/Trail  Intersection 

H.  Saddle 

I.  Spur/Finger 

J.  Stream/River 

K.  Stream/River  Intersection 

L.  Valley 


1.  _  2.  _  3.  _  4.  _  5. 
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A.  Depression 

B.  Draw 

C.  HUlTop 

D.  Lake/Pond 

E.  Ridge  Line 

F.  Road/Trail 

G.  Road/Trail  Intersection 

H.  Saddle 

I.  Spur/Finger 

J.  Stream/River 

K  Stream/River  Intersection 
L.  Valley 


A.  Depression 

B.  Draw 
C  Hill  Top 

D.  Lake/Pond 

E.  Ridge  Line 
E  Road/Trail 

G.  Road/Trail  Intersection 

H.  Saddle 

I.  Spur/Finger 

J.  Stream/River 

KL  Stream/River  Intersection 
L.  VaUey 


1.  _  2.  _  3.  _  4.  _  5. 
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Using  the  following  map  representations,  choose  the  best  representation  for  each 
picture  displayed  below.  The  map  representations  are  a  facsimile  of  the  terrain  shown  in 
the  photos.  Some  map  representations  may  be  used  more  than  once  or  not  at  all. 


6.  GUILFORD-ZIMMERMAN  APTITUDE  SURVEY 


The  Guillord-ZimmermanAptjtode  Survey 

Part  (Spatial  0il0ribti(ui 


t94rSf3&‘d3^S^Ce^  OtvsHfiBi,  CA 
OsifiJwasf  i?*  iSaiSirr^  »esj.  iw. 


Name 


Date  — : -  Score  — z -  Sex:  M  F 


INSTHUCnONS. 

This  is  a  test  of  your  abiiity  to  see  changes  in  cBrection  and  position.  In  each  item  you  are  to  note  how  the 
position  of  the  boat  has  aianged  in  the  second  picture  from  the  original  position  In  the  first  picture. 


Here  is  Sample  Item  t. 

These  bars  represent  the  boat's 
jyow. 


This  Is  the  correct  answer.  It  shows 
that  the  prow  of  the  boat  has  drop¬ 
ped  below  the  aiming  point. 


(If, the  j^'ow  had  risen,  instead  of 
dropped,  the  correct  answer  would 
have  been  C,  Instead  of  D.) 


"Hrese  are  the  five  possible  answers  to  the  Hem. 


This  is  the  prow  (Ironl  end)  <A  a 
motor  boat  in  which  you  are  riding. 

ThlsIs/me  airringT^int.  It  i^th^ 
exact  spot  you  would  se#bn  iarid 
if  you  sighted  right  over  the  .point 
oftiieprow. 


This  Is  the  same  aiming  point 
shown  above.  Note  that  the  prow 
has  drofped  below  it. 


To  work  each  item;  First,  look  at  thetop  picture  and  see  where  the  motor  boat  is  headed.  Second,  look  at  the 
bottom  picture  smd  note  the  CHANGE  in  the  boat's  heading.  Third,  mark  the  answer  that  shows>1he  same  change  on 
the  separate  answer  sheet. 


Try  Sample  Item  2. 

This  also  shows  that  the  prow  of 
the  boat  Is  to  the  right  of  the  aiming 
point.  So,  H  Is  the  correct  answer  - 


(If  the  boat  had  turned  to  the  left, 
instead  of  to  the  right,  the  correct 
answer  would  have  been  A.) 


Consulting  Psychologists  Pres^  Inc.,  S803  E  Bayshore  Road,  Palo  AHo,  CA  94303 
98  97  96  9S  34  87654 


0039 


Figure  E.l  Guilford-Ziimnerman  Aptitude  Survey  Cover  Page 
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7.  PRACTICE  MODEL  TEST 

a.  Turn  to  a  heading  of  360  degrees  and  begin  movement. 

b.  Switch  to  a  top  down  view 

c.  Switch  to  a  15-meter  view 

d.  Change  to  run  mode 

e.  Change  to  walk  mode 

f.  Move  to  the  road  and  take  a  right 

g.  While  following  the  road: 

i)  Look-up 

ii)  Look  down 

iii)  Look  left 

iv)  Look  Right 

h.  Head  into  town 

i.  Stop 

j.  What  is  your  heading? 

k.  Begin  movement. 

L  Run 

m.  Slow  down  and  stop  at  the  road  sign 

n.  Look  to  your  right.  What  do  you  see? 

o.  Using  the  quick  view  keys,  see  what  is  at  CP6 

p.  Using  the  hot  keys,  return  to  the  start  point 
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8.  DEBRIEFING  QUESTIONNAIRES 


a.  Map  Group  Debriefing  Questionnaire 


MAP  1 

Hard  to  Read 

Easy  to  Read 

Was  the  map  easy  to  read? 

1  2 

3 

4 

5  N/A 

Was  the  map  easy  to  understand? 

Hard  to  Understand 

1  2 

3 

4 

Easy  to  Understand 

5  N/A 

Were  the  trails  &  roads  adequately 
shown  on  the  map? 

Definitely  Not 

1  2 

3 

4 

Definitely  Yes 

5  N/A 

Were  the  man  made  structures 

Adequately  shown  on  the  map? 

Definitely  Not 

1  2 

3 

4 

Definitely  Yes 

5  N/A 

Were  the  obstacles  adequately 
shown  on  the  map? 

Definitely  Not 

1  2 

3 

4 

Definitely  Yes 

5  N/A 

Was  the  vegetati(Mi  adequately 
shown  on  the  map? 

Definitely  Not 

1  2 

3 

4 

Definitely  Yes 

5  N/A 

Using  the  map,  how  difficult  was  it 
to  plan  your  route? 

Easy 

1  2 

3 

4 

Very  Difficult 

5  N/A 

Comments: 

COURSE  1 

Easy 

Very  Challenging 

How  difficult  was  the  course? 

1  2 

3 

4 

5  N/A 

Were  the  control  points  well  marked? 

Definitely  Not 

1  2 

3 

4 

Definitely  Yes 

5  N/A 

Were  the  control  points  located 
where  you  expected  them? 

Definitely  Not 

1  2 

3 

4 

Definitely  Yes 

5  N/A 

Had  routes  been  trampled  down 
leading  to  the  control  points? 

Definitely  Not 

1  2 

3 

4 

Definitely  Yes 

5  N/A 

Did  you  have  difficulties  remembering 
your  plaimed  route? 

Definitely  Not 

1  2 

3 

4 

Definitely  Yes 

5  N/A 

Comments: 

MISC  1 

Definitely  Not 

Definitely  Yes 

Did  you  enjoy  this  experiment? 

1  2 

3 

4 

5  N/A 

Did  you  feel  the  training  phase 
was  long  aiough? 

Definitely  Not 

1  2 

3 

4 

Definitely  Yes 

5  N/A 

Did  you  feel  the  training  phase 
was  too  short? 

Definitely  Not 

1  2 

3 

4 

Definitely  Yes 

5  N/A 

Do  you  feel  the  training  familiarized 
you  learn  the  environmait? 

Definitely  Not 

1  2 

3 

4 

Definitely  Yes 

5  N/A 

Did  you  feel  confident  in  navigating 
the  tOTain  without  a  map  or  compass? 

Definitely  Not 

1  2 

3 

4 

Definitely  Yes 

5  N/A 

Comments: 

1.  Place  an  "X"  next  to  the  items  you  feel  must  be  replicated  in  a  model  that  prq)ares  you  to  navigate  an 
actual  piece  of  terrain. 


Buildings 

factory 

Buildings 

houses 

Buildings 

public  buildings 

Buildings 

shacks 

Buildings 

other 

Misc 

compass 

Misc 

road  signs 

Misc 

rock  piles 

Misc 

sand  bags 

Misc 

street  signs 

Misc 

the  sun 

Misc 

people 

Misc 

animals 

Misc 

sound 

Misc 

other 

Misc 

other 

Terrain 

clearings 

Terrain 

depressions 

Terrain 

hills 

Terrain 

knolls 

Terrain 

ridgelines 

Terrain 

spurs/fingCTS 

Terrain 

other 

roads 

dirt  roads 

roads 

foot  paths 

roads 

paved  roads 

roads 

trails 

roads 

other 

obstacles 

electric  lines 

obstacles 

pits/fox  holes 

obstacles 

shallow  ditches 

obstacles 

telephone  poles 

obstacles 

towers 

obstacles 

trenches 

obstacles 

other 

vegetation 

bushes 

vegetation 

flowers 

vegetation 

grass/weeds 

vegetation 

trees 

vegetation 

other 

water 

lakes 

water 

marsh  lands 

water 

ponds 

water 

puddles 

water 

swamps 

water 

other 

2.  From  the  list  of  items  in  question  #  1,  choose  and  rank  the  six  items  you  feel  are  the  most  important 
for  a  computa-  model  >^ch  will  be  used  to  prepare  an  individual  to  navigate  an  actual  piece  of  terrain. 


1 

2 

3 

4 

5 

6 
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b.  Virtual  Environment  Group  Debriefing  Questionnaire 


Hard  to  Read 

Easy  to  Read 

Was  the  map  easy  to  read? 

1  2 

3 

4 

5  N/A 

Was  the  map  easy  to  undwstand? 

Hard  to  Understand 

1  2 

3 

4 

Easy  to  Understand 
5  N/A 

Were  dte  trails  &  roads  adequately 
shown  on  the  map? 

Definitely  Not 

1  2 

3 

4 

Definitely  Yes 

5  N/A 

Were  the  man  made  structures 
Adequately  shown  on  the  map? 

Definitely  Not 

1  2 

3 

4 

Definitely  Yes 

5  N/A 

Were  the  obstacles  adequately 
shown  on  the  map? 

Definitely  Not 

1  2 

3 

4 

Definitely  Yes 

5  N/A 

Was  the  vegetation  adequately 
shown  on  the  map? 

Definitely  Not 

1  2 

3 

4 

Definitely  Yes 

5  N/A 

Using  the  map,  how  difficult  was  it 

Easy 

Very  Difficult 

to  plan  your  route? 

1  2 

3 

4 

5  N/A 

Comments: 

COURSE  I  Easy 

How  difficult  was  the  course?  1  2 


Definitely  Not 

Were  the  control  points  well  marked?  1  2 


Were  the  control  points  located  Definitely  Not 

where  you  expected  them? _ 1 _ 2 


Had  routes  been  trampled  down  Definitely  Not 

leading  to  the  control  points? _  1  2 


Did  you  have  difficulties  remembering  Definitely  Not 

your  planned  route?  1  2 


Conunents: 


Very  Challenging 
5  N/A 


Definitely  Yes 
5  N/A 


Definitely  Yes 
5  N/A 


Definitely  Yes 
5  N/A 


Definitely  Yes 
5  N/A 


MISC  1 

Did  you  enjoy  this  experiment? 

Definitely  Not 

1  2 

3 

4 

Definitely  Yes 

5  N/A 

Did  you  feel  the  training  phase 
was  long  enough? 

Definitely  Not 

1  2 

3 

4 

Definitely  Yes 

5  N/A 

Did  you  feel  the  training  phase 
was  too  shcBt? 

Definitely  Not 

1  2 

3 

4 

Definitely  Yes 

5  N/A 

Do  you  feel  the  training  familiarized 
you  leam  the  envircmment? 

Definitely  Not 

1  2 

3 

4 

Definitely  Yes 

5  N/A 

Did  you  feel  confident  in  navigating 
the  torain  without  a  map  or  compass? 

Definitely  Not 

1  2 

3 

4 

Definitely  Yes 

5  N/A 

Comments: 

MODEL  \  Definitely  Not 

Was  the  model  clear  and  viewable?  1  2 


Definitely  Not 

Did  the  model  coincide  with  the  map?  1  2 


Were  the  trails  &  roads  adequately  Definitely  Not 

rqjresented  in  the  model? _ 1 _ 2 


Were  flie  man  made  structures  Definitely  Not 

adequately  represented  in  (he  model? _ 1 _ 2 


Were  the  obstacles  adequately  Definitely  Not 

represented  in  the  model?  _ 1 _ 2 


Was  the  vegetation  adequately  Definitely  Not 

represented  in  file  model? _ 1 _ 2 


Were  changes  in  elevation  adequately  Definitely  Not 

represented  in  the  model? _  1 _ 2 


Did  the  model  help  you  identify  the  Definitely  Not 

control  points  within  the  last  50m?  _ 1 _ 2 


Did  the  model  help  you  identify  the  Definitely  Not 

general  area  of  the  control  points?  _ 1 _ 2 


Using  the  model,  how  difficult  was  it  Easy 

to  plan  your  route? _ 1 _ 2 


Do  you  feel  the  model  gave  you  an  Definitely  Not 

advantage  you  normally  wouldn't  1  2 

have  had?  _ 


Would  you  use  this  tool  if  it  wcto  Definitely  Not 

available  for  mission  planning?  1  2 


Would  you  use  this  tool  if  it  were  Definitely  Not 

available  for  mission  rehearsal?  1  2 


Would  you  use  this  tool  if  it  wore  Definitely  Not 

available  for  navigation  training?  1 _ 2 


Comments: 


Definitely  Yes 
5  N/A 


Definitely  Yes 
5  N/A 


Definitely  Yes 
5  N/A 


Definitely  Yes 
5  N/A 


Definitely  Yes 
5  N/A 


Definitely  Yes 
5  N/A 


Definitely  Yes 
5  N/A 


Definitely  Yes 
5  N/A 


Definitely  Yes 
5  N/A 


Very  Difficult 
5  N/A 


Definitely  Yes 
5  N/A 


Definitely  Yes 
5  N/A 


Definitely  Yes 
5  N/A 


Definitely  Yes 
5  N/A 


MODEL  INTERFACE _ 

Were  you  able  to  easily  move  throu^ 
the  model? 


Was  the  joystick  easy  to  use? 


Was  the  acceleration  lever  easy  to  use? 


Were  the  toggle  buttons  easy  to  use? 


Your  overall  felling  about  the  interface? 


as  the  15-minute  train-up  on  the 
initial  model  useful? 


Was  the  15-minute  train-up  on  the 
initial  model  enough  time  to  become 
familiar  with  the  interlace? 


Did  file  use  of  three  screois  cause 
any  confusion  whai  maneuvering? 


Comments: 


I  Confusing 

1  2 


Confusing 

1  2 


Confusing 

1  2 


Confusing 

1  2 


Confusing 

1  2 


Definitely  Not 
1  2 


Definitely  Not 
1  2 


Definitely  Not 
1  2 


User  Friendly 
5  N/A 


User  Friendly 
5  N/A 


Usa-  Friendly 
5  N/A 


User  Friendly 
5  N/A 


User  ftiendly 
5  N/A 


Definitely  Yes 
5  N/A 


Definitely  Yes 
5  N/A 


Definitely  Yes 
5  N/A 


1.  Place  an  "X"  next  to  the  items  you  feel  must  be  replicated  in  a  model  that  prepares  you  to  navigate  an 
actual  piece  of  terrain. 


Buildings 

factory 

roads 

dirt  roads 

Buildings 

houses 

roads 

foot  paths 

Buildings 

public  buildings 

roads 

paved  roads 

Buildings 

shacks 

roads 

trails 

Buildings 

other 

roads 

other 

Misc 

compass 

obstacles 

electric  lines 

Misc 

road  signs 

obstacles 

pits/fox  holes 

Misc 

rock  piles 

obstacles 

shallow  ditches 

Misc 

sand  bags 

obstacles 

telephone  poles 

Misc 

street  signs 

obstacles 

towers 

Misc 

the  sun 

obstacles 

trenches 

Misc 

people 

obstacles 

other 

Misc 

animals 

Misc 

sound 

Misc 

other 

vegetation 

bushes 

Misc 

Other 

vegetation 

flowers 

vegetation 

grass/weeds 

Terrain 

clearings 

vegetation 

trees 

Terrain 

depressions 

vegetation 

other 

Terrain 

hills 

Terrain 

knolls 

Terrain 

ridgelines 

water 

lakes 

Terrain 

spurs/fingCTs 

water 

marsh  lands 

Terrain 

other 

water 

ponds 

water 

puddles 

water 

swamps 

water 

other 

2.  From  the  list  of  items  in  question  #  1,  choose  and  rank  the  six  items  you  feel  are  the  most  important 
for  a  computer  model  which  will  be  used  to  prepare  an  individual  to  navigate  an  actual  piece  of  terrain. 


1 

i 

i 

4 

s' 

6 
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APPENDIX  F.  COURSE 

1.  GENERAL 

This  appendix  consists  of  six  items:  the  1:24,000  scale  map  enlarged  to  1:5,000 
scale,  an  aerial  photo  of  the  course,  an  aerial  photo  with  an  exan:q)le  participant 
debriefing  route,  1:5,000  course  orienteering  map,  and  an  explanation  of  the  map  legend 
[BANK  97].  The  1:50,000  and  1:24,000  maps  are  the  standard  scales  used  by  most  US 
ground  forces  for  military  operations.  The  difference  in  detail  between  the  1:5,000  scale 
map  and  the  1:24,000  map  are  obvious  upon  conq)arison.  The  aerial  photo  is  the  same 
given  to  the  low  fidelity  map  subjects.  The  1:5,000  scale  orienteering  map  was  created  by 
Banker  [BANK  97]  and  modified  by  Goerger  [GOER  98b].  The  map  legend  explanation 
is  taken  directly  from  Appendix  D  of  MAJ  Banker’s  1997  Masters  Thesis. 
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AP  DATA  f: 


4.  AERIAL  PHOTO 


5.  AERIAL  PHOTO  WITH  PARTICIPANT  ROUTE 


Figure  F.5.  Aerial  Photo  With  Subject  Route 
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7.  COURSE  MAP  LEGEND  EXPLANATION 

All  maps  are  generalizations.  They  use  symbols  to  portray  actual  features  on  the 
earth’s  surface.  Not  all  features  are  represented  with  the  same  precision.  Discrete  non¬ 
vegetation  items  are  plotted  on  the  map  in  the  exact  location  they  are  in  the  actual 
environment,  whereas  vegetation  boundaries  (unless  indicated  with  a  distinctive  dotted 
line)  are  not  meant  to  represent  a  clean  break  from  one  type  of  vegetation  to  another. 
Rather,  this  line  separating  one  vegetation  area  from  another  is  a  generalization  of  where 
one  type  more  or  less  ends  and  another  more  or  less  begins.  The  line  separating  the  two 
can  best  be  thought  of  as  a  blurry  line  where  the  two  types  of  vegetation  intermingle.  The 
below  guide  will  help  to  determine  the  specific  limitations  of  each  symbol  on  the 
orienteering  map. 

Building  -  Buildings  in  the  area  are  of  several  types: 

a.  Latrines  -  most  common  building,  tan  in  color,  approx,  size  3x8  meters 

b.  Shelters  -  second  most  common  building,  green  wood,  roofed,  no  walls, 

approx,  size  3x8  meters 

c.  Admin.  -  field  office  and  shack,  black  with  gold  trim,  8x8  meters  and  2x2 

meters  respectively 

Open  Sandy  Ground  -  a  significant  patch  of  sand  that  will  slow  running 

Open  ground  -  dirt,  hard  pack,  free  of  grass  and  other  vegetation. 

Undergrowth  walk  -  immature  chaparral  or  oak,  dense  stands  of  bushes, 
incon^lete  overlap  of  two  distinct  areas  of  fight  which  allow  restricted  passage  along  that 
overlap,  other  plants  that  prevent  running. 
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Fight  -  mature  chaparral  or  immature  oak  in  such  density  that  passage  through  is 
very  difficult,  running  impossible 

Forest  walk  -  oak  forest  with  patchy  undergrowth,  low  lying  tree  limbs  or  tree 
density  that  prevents  running  from  being  sustained 

Forest  slow  run  -  oak  forest  fairly  free  of  undergrowth,  but  with  low  lying  limbs 
or  tree  density  that  makes  sustained  running  difficult. 

Rough  open  ground  -  grass  covered  ground,  possibly  with  scattered  (avoidable) 
undergrowth.  Note  that  there  are  a  few  locations  that  have  what  appears  to  be  old  jeep 
trails  but  are  portrayed  as  rough  open  ground.  Sometimes  the  distinction  between  one  or 
the  other  blurs.  If  in  doubt  refer  to  other  noore  distinctive  features  (contour  lines,  etc.)  to 
determine  your  location. 

Shallow  depression  -  most  likely  an  old  decaying  foxhole  position  or  other  man 
made  excavation  where  the  banks  have  eroded  to  create  a  bowl-like  depression  of  1  to  3 
feet  below  surrounding  ground. 

Misc.  object  -  a  manmade  feature,  rubble,  derelict  military  equipment,  or  other 
item  whose  exact  description  is  only  provided  if  it  is  the  location  of  a  control 

Pit  -  an  old  foxhole  or  likely  other  man  made  pit  that  has  steep  vertical  walls  and 
may  be  reinforced  with  wood,  depth  from  2  to  5  feet.  Note  that  there  will  be  many  pits  in 
the  area  that  are  not  depicted  on  the  map.  The  pits  that  are  depicted  are  accurate. 

Telephone  poles  -  wood  poles  (if  bearing  wire  it  will  be  noted  on  map)  approx. 
25  to  30  feet  in  height 

Concrete  pad  -  old  concrete  tent  pad  extending  from  2  to  5  inches  above  ground 

level 
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Tree  -  a  tree  or  large  bush  (could  be  two  or  more  trees  growing  close  together  - 
forming  an  unbroken  single  canopy  —  if  the  trees  are  small) 

Rootstock  -  a  dead  or  overturned  tree 

Troop  training  device  -  a  bunker  or  other  man  made  item  built  for  training 
soldiers 

Vegetation  boundary  -  the  edge  of  a  vegetation  type 

Gully  or  Ditch  -  ranging  from  a  shallow  1-foot  deep  gully  to  5-foot  deep  military 

trench 

Jeep  Trail  -  a  road  more  suitable  for  4  x  4  vehicles  due  to  width  restriction  and/or 
ruts.  May  be  distinctive  and  worn  or  in  some  places  overgrown  with  grass  but  still 
containing  ruts. 

Paved  Road  -  a  surfaced  all  weather  road 

Road  -  a  sandy  or  dirt  toad  wide  and  level  enough  for  2  wheel  drive  vehicles 

Indistinct  Path  -  a  path  that  is  in  the  process  of  being  overgrown  with  only 
intermittent  marks  on  the  ground  that  indicate  that  it  was  once  a  well  traveled  path 

Narrow  Ride  -  a  linear  break  in  the  forest  that  may  have  once  been  a  jeep  trail 
but  now  is  overgrown  with  grass  and  lacks  telltale  wheel  ruts 

Path  -  a  foot  or  bike  path. 
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8.  CLUE  SHEET 
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APPENDIX  G.  PARTICIPANT  TASK  LIST^ 

Thank-you  for  participating  in  this  study.  You  will  do  an  Orienteering  course 
today.  However,  there  are  some  important  differences  to  note: 

1.  You  will  be  wearing  a  light  pack  with  DGPS  and  Newton  MSG  Pad  130.  Its  purpose 
is  to  log  your  route  and  act  as  a  data  capture  device  for  other  actions  you  may  perform. 

2.  Before  you  run  the  course  you  will  carefully  plan  your  route  through  the  entire  course 
(see  In^ortant  Information  on  Marking  Your  Map) 

3.  Use  this  training  time  to  commit  the  route  and  course  to  memory.  You  are  expected  to 
do  the  following  on  the  actual  course  run: 

a.  Navigate  without  aid  of  map  and  compass,  utilizing  only  your  memory 

b.  Atten^t  to  find  all  the  controls  utilizing  your  planned  route 

Summary  of  objectives 
All  Objectives  are  equally  important!! 

1.  Choose  the  most  efficient  route  based  on  your  abilities 

2.  Minimize  the  number  of  map  checks  you  request  from  the  administrator 

3.  Minimize  the  number  of  con:q)ass  checks  you  request  from  the  administrator 

4.  Minimize  the  number  of  map  with  compass  checks  you  request  from  the 
administrator 

5.  Stay  on  your  planned  route 

6.  Find  all  the  controls  in  order  (you  have  60  minutes  to  conduct  this  task) 

•  If  you  need  to  make  a  map  check  then  say  so  and  the  administrator  will  give  you  the 
map  for  (30/60)  seconds.  Additional  time  can  be  requested  in  (30/60)  second 
increments  at  the  additional  cost  of  a  map  check  each. 

•  If  you  need  to  make  a  con^ass  check  then  say  so  and  the  administrator  will  give  you 
the  con^ass  for  30  seconds.  Additional  time  can  be  requested  in  30-second 
increments  at  the  additional  cost  of  a  con^ass  check  each. 

•  If  you  need  both  map  and  con5)ass  then  say  so  and  the  administrator  will  give  you 
both  for  (60/90)  seconds.  Additional  time  can  be  requested  in  increments  of  (60/90) 
seconds. 

•  If  you  want  to  change  your  route  announce  to  the  administrator  that  you  are  changing 
your  route  plan.  At  that  point  the  administrator  will  hand  you  the  map,  con5)ass,  and 
blue  pen.  From  the  time  that  he  gives  you  the  materials  you  will  have  60  seconds  to 
plot  the  new  route.  If  you  need  more  time  then  teU  him  you  need  more  time  and  you 
will  get  another  60  seconds.  Request  additional  time  as  needed  but  remember  that  one 
of  your  objectives  is  to  make  as  few  map  checks  as  necessary.  Every  (30/60)  seconds 


^  This  document  is  adapted  and  modified  from  MAT  Banker’s  Masters  Thesis  [BANK  98] 
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that  you  are  looking  at  the  map  heyond  the  original  60  seconds  for  the  route 
change  counts  as  a  map  check. 

Note  that  when  two  times  are  given  (30/60)  the  larger  time  applies  to  low  fidelity  map 
subjects  since  they  had  both  the  map  and  the  aerial  photograph  to  consult. 
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APPENDIX  H,  MAP  MARKING  INSTRUCTIONS^ 

Pay  close  attention  to  how  you  mark  your  route,  be  as  precise  as  the  map  and  pen 
allow.  Before  your  actual  run  you  are  expected  to  preview  your  map  within  your  group’s 
prescribed  context.  Mark  your  planned  route  using  the  RED  pen.  You  may  correct  any 
mistakes  you  make  while  planning  with  the  white  eraser.  Once  the  planning  period  is  up 
or  you  elect  to  finish  you  will  not  be  allowed  to  erase  any  of  the  red  route  marks  you  have 
made.  SO  BE  PRECISE  in  marking  your  map,  detail  does  matter.  Later  during  the 
actual  course  run  anytime  that  you  are  going  to  deviate  from  your  planned  route  you  must 
stop: 

1.  Announce  to  the  administrator  that  you  are  changing  your  route  plan.  At  that  point  the 
administrator  will  hand  you  the  map.  From  the  time  that  he  gives  you  the  map  you  will 
have  60  seconds  to  plot  the  new  route.  If  you  need  more  time  than  tell  him  you  need  more 
time  and  you  will  get  another  (30/60)  seconds.  Request  additional  time  as  needed  but 
remember  that  one  of  your  objectives  is  to  make  as  few  map  checks  as  necessary.  Every 
(30/60)  seconds  that  you  are  looking  at  the  map  beyond  the  original  60  seconds  for 
the  route  change  counts  as  a  map  check. 

2.  Take  the  blue  pen  and  draw  in  your  new  route  with  the  same  attention  to  detail  that  you 
apphed  or  the  original  route  planning  in  red. 

3.  Leave  your  original  route  on  the  map.  The  eraser  is  provided  so  that  you  may  make 
corrections  to  a  route  as  you  draw  it.  Once  you  finish  drawing  and  begin  navigating  you 
are  not  allowed  to  erase  routes,  or  corrections  to  plaimed  routes  (blue  permed  routes). 

4.  You  may  make  as  many  corrections  to  your  route(s)  as  necessary  while  navigating  the 
course. 


Importance  of  detail  in  map  marking  and  navigation 

You  are  allowed  to  deviate  fi’om  your  plarmed  route  within  the  following 
tolerances  while  still  being  considered  on  that  route: 

Jeep  Trails,  Paved  Roads,  Unpaved  Roads,  Indistinct  Paths,  Narrow  Rides  and 
Paths  —  If  your  marked  route  is  on  any  of  these  features  you  are  allowed  5  meters  either 
side  of  the  feature  and  you  are  still  considered  as  being  "on  your  route". 

All  other  features  -  On  all  other  types  of  non  road/trail  terrain  you  may  travel  15 
meters  to  either  side  of  your  marked  route  and  you  are  still  considered  as  being  "on  your 
route" 


*This  document  is  adapted  and  modified  from  MAJ  Banker’s  Masters  Thesis  [BANK  98] 
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APPENDIX  1.  DIGITAL  PHOTOS 


1.  GENERAL 

As  part  of  their  training  aids,  subjects  are  provided  with  a  series  of  digital  images 
of  the  control  points.  Map-only  subjects  received  the  photos  displayed  in  Appendix  1.2 
while  Virtual  Environment  subjects  receive  the  Appendix  1.3  photo  sets.  The  VE  subjects 
also  received  screen  shot  images  from  the  VE  of  what  the  control  looks  like  in  the  model. 
This  is  so  the  subject  can  con:q>are  the  model  to  the  real  world.  The  photos  are  furnished 
in  color.  Having  the  corresponding  VE  screen  shot  helps  VE  subjects  overcome  some 
model  deficiencies,  such  as  lack  of  accurate  ground  cover,  and  lack  of  negative  spot 
elevations  (i.e.,  pits  and  ditches)  in  the  model.  The  photographs  provide  a  useful  tool  for 
the  subject  in  fixing  exactly  what  the  target  control  point  looks  like,  and  perhaps  some  of 
what  the  surrounding  terrain  looks  like.  Having  this  type  of  information  is  not  entirely 
unreasonable  for  a  military  mission.  The  photos  help  to  outfit  the  subject  with  a  stronger 
grasp  of  the  defining  landmarks  they  are  searching  for. 
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2.  MAP  AND  REAL  WORLD  GROUP  PHOTOS 


Control  Point  1  Control  Point  4 


Control  Point  2  Control  Point  5 


Control  Point  7 


Control  Point  8 
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APPENDIX  J.  COURSE  EQUIPMENT  CHECKLIST 


Binder  Containing: 

_ Subject’s  map  &  designated  route 

_ Think  Out  Loud  Instructions 

_ Data  Collection  Sheet 

_ Researcher’s  Script 

Data  Recording: 

_ blue  alcohol  pen  to  record  route  deviations 

_ red  pen  to  record  data 

_ digital  camera 

_ helmet  &  8mm  camera 

_ rucksack  frame  w/GPS  system 

_ stop  watch/timer 

Misc: 

_ extra  battery  (8nmi  camera) 

_ extra  cassette  (8mm  camera) 

_ extra  Color  Wheels  for  Tasks  3.1.  &  5.1 

_ extra  arrows  (color  wheels) 

_ extra  clue  sheet  (incase  subject  looses  his/hers) 

_ blindfold  (for  movement  to  course) 

_ cellular  phone  {optional) 

_ con^ass 

_ first  aid  kit 

_ Tecnu  (for  poison  oak) 

_ water 

Prepositioned: 

_ Color  Wheel  Platform  for  Tasks  3.1.  &  5.1 

_ Control  flags 
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APPENDIX  K.  THINK  OUT  LOUD  INSTRUCTIONS* 

Your  thoughts  are  important  to  this  research.  As  you  navigate  the  course,  you 
should  be  “thinking  out  loud”. 

As  you  move  through  the  environment  and  experience  it  directly,  express  what 
you  are  thinO^g.  The  mental  preconception  you  had  of  this  environment  before  you 
stepped  into  it  will  now  be  evaluated  by  you  as  you  experience  the  course  directly.  As 
this  image  is  confronted  with  direct  experience  your  expectations  and  plan  may  be 
confirmed,  modified,  or  refilled.  Be  sure  to  talk  “out  loud”  these  thoughts. 

The  process  of  talking  “out  loud”  and  paying  close  attention  to  your  route  will 
slow  you  down.  This  is  expected  and  why  you  are  given  an  hour  to  finish  the  course. 


PLEASE  SPEAK  LOUDLY  SO  THAT  YOUR  VOICE  WILL  BE  PICKED  UP  BY 
THE  MONITOR  AND  HE  CAN  RECORD  WHAT  YOU  SAY. 


*  This  document  is  adapted  and  modified  from  MAT  Banker’s  Masters  Thesis  [BANK  98] 
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APPENDIX  L.  ROUTE  CLASSIFICATIONS 

1.  GENERAL 

This  appendix  consists  of  five  items:  route  analysis,  an  explanation  of  route 
classifications  for  each  leg  of  the  course  [BANK  97],  and  route  classifications  based  on  a 
Goerger’s  LISP  Program  [GOER  98b].  The  explanation  of  route  classifications  for  each 
leg  of  the  course  is  taken  directly  from  Appendix  F  of  MAJ  Banker’s  1997  Masters 
Thesis.  The  explanation  of  the  LISP  routes  is  taken  directly  from  MAJ  Goerger’s  1998 
Master  Thesis.  Route  classifications  were  utilized  to  categorize  the  difficulty  of  an 
individual’s  planned  routes  for  comparison  to  their  navigational  ability.  Routes  were 
classified  using  MAJ  Banker’s  route  classification  listing  and  again  utilizing  the  results  of 
Goerger’s  LISP  route  planning  program 

2.  ROUTE  ANALYSIS 

Participant  routes  were  analyzed  for  difficulty  level  and  performance. 
Partic^ants’  Leg  Error  Scores  were  correlated  with  their  Leg  Difficulty  Rating  and 
ability  level.  The  relationship  between  a  subject’s  route  difficulty  and  their  overall 
performance  is  discussed  in  Chapter  IV,  Section  B.3. 

Goerger  felt  that  if  participants  planned  routes  which  were  beyond  their  ability 
level  (a  novice  planning  expert  routes),  the  chances  they  will  fail  to  successfully  execute 
the  planned  routes  increases.  Advanced  routes,  while  being  faster  and  perhaps  more 
direct,  required  true  expertise  in  navigational  skills  to  successfully  implement  them  In 
fact,  most  “expert”  navigators  did  not  plan  “expert”  routes.  The  poorest  routes  were 
planned  by  novices  who  failed  to  realize  what  exactly  they  were  attenpting. 

3.  BANKER’S  ROUTE  CLASSIFICATIONS 

What  foUows  is  MAJ  Banker’s  classification  of  some  of  the  most  probable  routes 
to  a  given  control  and  is  based  on  the  International  Specification  for  Orienteering  Maps 
[INTE  90].  They  do  not  represent  the  only  ways  of  getting  to  a  control  but  the  most  likely 
routes  chosen  by  participants  based  upon  MAJ  Banker’s  orienteering  experience  and 
knowledge  of  the  terrain.  The  classifications  are  used  as  a  basis  for  con^arison  with  the 
routes  selected  by  Goerger’s  LISP  Route  Selection  Program  Note  that  all  controls 


171 


possess  at  least  one  beginner’s  route.  If  a  participant  did  not  exactly  follow  the  route 
described  by  Banker,  the  route  was  examined.  If  it  had  numerous  handrails,  it  was 
considered  intermediate.  If  it  had  more  catching  features,  it  was  considerd  advanced.  A 
lack  of  any  navigational  features,  or  a  direct  line  azimuth  between  any  but  the  closest 
points  earned  an  advanced  rating  [GOER  98b]. 

a.  Control  1. 

1.  Beginner 

a)  Gigling  Road  west  to  jeep  trail 

b)  Jeep  Trail  south  by  east  by  south  to  building 

c)  Control  on  NW  comer  of  building 

2.  Beginner 

a)  Watkin's  Gate  Cutoff  to  indistinct  path. 

b)  Indistinct  path  southwest  up  hill  to  jeep  trail 

c)  Jeep  Trail  west  to  building 

d)  Control  on  NW  comer  of  building 

3.  Intermediate 

a)  West  through  plotted  individual  trees  (catching  features) 

b)  Handrail  rough  open  ground  south  to  junction  indistinct 
path  and  jeep  trail 

c)  Jeep  Trail  west  to  budding  (catching  feature) 

d)  Control  on  NW  comer  of  building 

4.  Advanced 

a)  West  through  plotted  individual  trees 

b)  Follow  runnable  forest  southwest 

c)  Try  to  hit  small  rough  open  gap  by  keeping  walkable  forest 
to  left  shoulder 

d)  Use  forest  fight  to  west  as  catching  feature  if  needed 

e)  Control  on  NW  comer  of  building 

f)  Use  jeep  trail  for  catching  feature  if  control  is  missed 

5.  Advanced 

a)  Go  straight  at  control  from  start 

b.  Control  2. 
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1.  Beginner 

a)  Jeep  trail  northwest  to  building 

b)  Follow  open  ground  to  west  and  look  for  rough  open 
clearing  going  northwest  (handrail) 

c)  Follow  rough  open  clearing  northwest  looking  for  pit 

d)  Control  in  pit 

2.  Intermediate 

a)  Jeep  trail  northwest  to  building 

b)  Go  straight  at  control  (WSW)  from  building 

3.  Advanced 

a)  Set  out  on  straight  line  directly  for  control 

b)  Hit  open  ground  and  look  for  building  on  the  right  and 

rough  open  break  on  the  left.  (Catching  feature) 

c)  Follow  rough  open  clearing  northwest  looking  for  pit 

d)  Control  in  pit 

Control  3. 

1.  Beginner 

a)  Head  northwest  and  get  out  onto  Gigling  Road 
.  b)  Take  Gigling  Road  west  to  jeep  trail  junction  with 
telephone  pole 

c)  Take  jeep  trail  southeast  to  convergence  of  two  jeep  trails 

d)  Head  southwest  into  tree  grove  looking  for  control 

(1)  Use  building  as  catching  feature 

(2)  Use  open  ground  to  west  as  backup  catching  feature 

e)  Control  hanging  from  tree  limb 

2.  Advanced 

a)  Head  straight  at  control;  use  jeep  trail  prior  to  control  as 
catching  feature 

b)  Head  southwest  into  tree  grove  looking  for  control 

(1)  Use  building  as  catching  feature 

(2)  Use  open  ground  to  west  as  backup  catching  feature 

c)  Control  hanging  from  tree  limb 
Control  4. 
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e. 


1.  Beginner 

a)  Head  southwesterly  and  try  to  get  on  jeep  trail  headed  in 
same  direction 

b)  Take  jeep  trail  to  junction 

c)  T ake  jeep  trail  southeast  to  junction 

d)  Take  southerly  fork  to  next  junction 

e)  Take  fork  to  northwest 

f)  Once  beyond  patches  of  fight  leave  trail  and  start  looking 
for  control 

g)  Control  is  in  pit 

2.  Beginner 

a)  Turn  around  and  go  back  to  jeep  trail  to  the  east 

b)  T ake  jeep  trail  southwest  to  junction 

c)  Take  fork  to  the  south  to  another  junction 

d)  Take  fork  to  the  west  to  next  junction 

e)  Take  southerly  fork  to  next  junction 

f)  Take  fork  to  northwest 

g)  Once  beyond  patches  of  fight  leave  trail  and  start  looking 
for  control 

h)  Control  is  in  pit 

3.  Intermediate 

a)  Go  south  towards  road  junction 

b)  Get  on  road  and  take  to  junction 

c)  Take  road  west  to  other  road  junction 

d)  Handrail  around  fight  to  west  coming  down  through  small 
patch  of  fight  into  control 

4.  Advanced 

a)  Head  straight  at  control  expect  to  hit  jeep  trail  that  runs 
NW  to  SE  (catching  feature) 

b)  Hit  trail  and  then  thread  way  through  scattered  fight 

c)  Emerge  into  center  of  depression  and  rough  open  ground, 

(catching  feature)  look  for  pit 

d)  Control  is  in  pit 

Control  5. 

1.  Beginner 


174 


a)  Move  back  out  onto  jeep  trail 

b)  Take  trail  west  to  trail  junction 

c)  Take  trail  WNW  up  to  misc  object 

d)  From  misc.  object  go  straight  at  control 

2.  Intermediate 

a)  Move  directly  at  control 

b)  Use  Gigling  Road  as  catching  feature  if  miss  on  control 

c)  Control  is  in  center  of  clearing 

3.  Advanced 

a)  Move  directly  at  control 

b)  Use  southwesterly  linear  clearing  as  catching  feature 

c)  Follow  clearing  1^  right  into  control 

d)  Use  Runnable  forest  along  Gigling  as  catching  feature  in 
case  of  miss 

Control  6. 

1.  Beginner 

a)  Move  out  onto  Gigling  Road  and  take  it  westerly  to 
junction  with  dirt  road 

b)  Move  down  dirt  road  (south)  to  junction  with  jeep  trail 

c)  Take  jeep  trail  to  east  look  for  concrete  rubble 

d)  Move  southeast  through  runnable  forest 

e)  Look  for  control  on  concrete  pad 

2.  Beginner 

a)  Move  straight  at  control  and  hit  jeep  trail 

b)  Go  southwest  on  Jeep  trail  to  junction  with  another  jeep 

trail 

c)  Take  jeep  trail  westerly  and  look  for  concrete  rubble 

d)  Move  southeast  through  runnable  forest 

e)  Look  for  control  on  concrete  pad 

3.  Intermediate 

a)  Move  south  to  junction  of  two  jeep  trails  (catching  feature) 

b)  Handrail  jeep  trail  southeasterly  to  clearing  (catching 

feature) 

c)  Handrail  clearing  to  the  west 

d)  Hit  fight  going  west  (catching  feature)  and  move  south 
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e)  Handrail  fight  (keeping  it  on  right  shoulder)  into  control 

f)  Look  for  control  on  concrete  pad 

4.  Advanced 

a)  Move  straight  at  concrete  rubble  (aiming  off  technique)  use 
jeep  trail  as  catching  feature  and  handrail 

b)  Move  southeast  through  runnable  forest 

c)  Look  for  control  on  concrete  pad 

g.  Control  7. 

1.  Beginner 

a)  Move  back  out  onto  east  west  jeep  trail 

b)  Go  west  to  junction  of  jeep  trail  and  dirt  road 

c)  Take  dirt  road  south  to  junction  with  four  jeep  trails 

d)  Take  jeep  trail  east  by  northeast 

e)  Look  for  second  linear  break  in  vegetation  (indistinct  path) 

f)  Take  indistinct  path  (handrail)  to  ditch 

g)  Follow  ditch  to  its  end 

h)  Control  at  east  end  of  ditch 

2.  Intermediate 

a)  Move  through  rough  open  ground  easterly  to  jeep  trail 
(catching  feature) 

b)  Follow  jeep  trail  (handrail)  to  junction  with  other  jeep  trail 
by  building 

c)  Locate  telephone  poles  and  follow  wire  (handrail)  south 
easterly 

d)  Hit  fight  and  turn  west  and  follow  fight  boundary  into  ditch 
(handrail) 

e)  Control  at  east  end  of  ditch 

3.  Advanced 

a)  Move  through  rough  open  ground  easterly  to  jeep  trail 
(catching  feature) 

b)  T ake  jeep  trail  to  curve  where  it  turns  east  (hand  rail) 

c)  Leave  jeep  trail  and  head  straight  for  control  use  east  west 
jeep  trail  as  checkpoint  (catching  feature) 

d)  Aim  off  to  east  side  of  ditch  and  go  southeast  (telephone 
wires  to  east  as  catching  feature  to  prevent  drifting  too  far 
east) 

e)  Use  fight  as  catching  feature 
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f)  Hit  fight  and  turn  west  and  follow  fight  boundary  into  ditch 

g)  Control  at  east  end  of  ditch 

4.  Advanced 

a)  Move  straight  at  control 

b)  Use  jeep  trail  junction  as  attack  point 

c)  From  attack  point  take  offeet  route  to  west  part  of  ditch 

d)  Follow  ditch  to  east  and  find  control  at  end  of  ditch 

h.  Control  8. 

1.  Beginner 

a)  Handrail  fight  to  the  east  till  hitting  the  jeep  trail 

b)  Follow  jeep  trail  northerly  through  intersection  to  sharp 
curve  to  the  east 

c)  Once  at  sharp  curve  to  east  turn  off  trail  to  west  and  look 
for  control  in  clearing 

d)  Control  located  in  clearing 

2.  Intermediate 

a)  Handrail  fight  to  telephone  poles 

b)  Take  telephone  poles  NW  back  to  jeep  trail  junction 

c)  Follow  jeep  trails  east  to  next  junction 

d)  Take  jeep  trail  north 

e)  Leave  jeep  trail  and  move  directly  at  control 

3.  Advanced 

a)  Move  directly  at  control  (avoiding  forest  walk)  use  jeep 
trail  junction  as  catching  feature 

b)  From  jeep  trail  junction  aim  off  to  east  of  control  at  sharp 
curve  to  east  of  jeep  trail  keeping  eyes  open  for  control  in 
clearings 

c)  Use  same  trail  as  Beginner  route  as  catching  feature  (for 
drift) 

L  Control  9  (Finish) 

1.  Beginner 

a)  Move  back  out  to  jeep  trail  just  to  east  of  control  8 

b)  Take  trail  south  to  four  way  junction  with  other  trails 
(handrail) 
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c)  Take  southeasterly  running  trail  to  trail  fork 

d)  Take  northeasterly  running  fork  to  five  way  junction 
(handrail) 

e)  Take  northwesterly  running  trail  keeping  eyes  open  for 
small  break  in  fight  to  the  east  (catching  feature) 

f)  Take  indistinct  path  into  clearing  and  hook  to  north 

g)  Control  on  east  edge  of  clearing 

2.  Intermediate 

a)  Move  back  out  to  jeep  trail  just  to  east  of  control  8 

b)  Move  off  trail  using  rough  open  to  move  closer  to  control 

c)  Take  rough  open  out  onto  jeep  trail  which  runs  ME  to  SW 

d)  Take  trail  to  junction  with  North  South  jeep  trail 

e)  follow  jeep  trail  looking  for  indistinct  path 

f)  Take  indistinct  path  into  clearing  and  hook  to  north 

g)  Control  on  east  edge  of  clearing 

3.  Advanced 

a)  Move  straight  at  control  on  east  by  northeast  azimuth 

b)  Use  trail  as  catching  feature 

c)  Fight  to  north  and  south  of  route  used  as  catching  features 

d)  Locate  opening  in  fight 

e)  Take  indistinct  path  into  clearing  and  hook  to  north 

f)  Control  on  east  edge  of  clearing 

4.  LISP  PROGRAM  ROUTE  CLASSIFICATION 

This  section  is  taken  directly  firomMAJ  Goerger’s  thesis  [GOER  98b].  This  LISP 
program  plans  a  route  through  a  specified  piece  of  terrain  based  on  identifiable  decision 
points  and  terrain  characteristics.  The  information  is  manipulated  by  a  branch  and  bound 
search,  pruning  heuristics,  and  terrain  classification. 

The  program  is  designed  to  locate  three  optimal  paths  through  the  course.  One 
Beginner  (Figure  L.l),  one  Intermediate  (Figure  L.2),  and  one  Advanced  Course  (Figure 
L.3)  are  calculated  and  displayed  on  maps  for  con^arison  with  participant  maps.  The 
program  also  produces  a  sequential  list  of  decision  points  or  waypoints  to  traverse  in 
order  to  coaq)lete  the  course.  Each  leg  of  a  participant’s  route  is  compared  to  the  LISP 
program  route  legs.  If  two  LISP  routes  have  legs  that  are  the  same,  the  leg  is  classified  as 
the  easier  of  the  two  routes.  If  a  participant’s  planned  route  between  control  points  is  not 
the  same  as  any  of  the  con^uter  program’s  planned  routes,  the  participant’s  route  is 
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assigned  a  classification  which  is  most  closely  associated  with  the  participant’s  route  with 
respect  to  the  program  algorithm’s  defining  characteristics. 


Figure  L.  1 .  LISP  Beginner  Route 


Fort  Old  Land  Navigation  Map 


Figure  L.2.  LISP  Intermediate  Route 
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Figure  L.3.  LISP  Advanced  Route 


APPENDIX  M.  DATA  COLLECTION  WORKSHEETS 


1.  TRAINING  PHASE  DATA  COLLECTION  SHEET 


Initial  Subject  Study 
Method: 


Session  Date: 
Sessiim  Start  Time: 
Sessimi  End  Time: 


a)  Study  Map  b)  Read  Map  and  Start  Mvt  c)  Eiqilore  Terrain 


Number  Compass  Checks: 


Number  Map  Checks: 


Number  of  times  subject  became  "lost": 


Number  of  times  subject  went  out  of  bounds  or  fell  off  the  edge  of  the  model: 


Did  the  subject  have  difficully  reading  the  compass 


Did  the  subject  have  difficulty  reading  the  map' 


Did  the  subject  have  difficulty  with  the  model  interface? 
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2.  EVALUATION  PHASE  DATA  COLLECTION  SHEET 


Session  Date: 

_ JSession  Start  Hme: 

Session  End  Time:  _ 


APPENDIX  N.  PARTICIPANT  DATA 

1.  GENERAL 

Subject  data  consists  of  two  items:  map  with  planned  route,  and  the  map  with 
executed  route.  For  subjects  who  had  the  low  fidelity  map,  their  planned  and  executed 
routes  are  shown  on  the  overhead  photo.  The  errors  for  deviation  fi"om  the  planned  route 
are  located  in  Appendix  O.  The  angle  and  distance  measurements  for  the  Wheel  and 
White  Board  Tests  can  be  found  in  Appendix  0.2  and  0.3  respectively. 
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2.  PARTICIPANT  NUMBER  Ml-1 


Figure  N.l.  Ml-1  Planned  Route 
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3.  PARTICIPANT  NUMBER  Ml-2 


Figure  N.3.  Ml-2  Planned  Route 
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4.  PARTICIPANT  NUMBER  Ml-3 


Figure  N.5.  Ml-3  Planned  Route 
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Field  Checked  by  Captian  Simon  R.  Goerger  USA 
May  1998 


Figure  N.7.  Ml -4  Planned  Route  / 

•? 
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Participant  Number 
Weather  _ 


6.  PARTICIPANT  NUMBER  Ml-5 


Figure  N.9.  Ml-5  Planned  Route 
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Ft  Ord,  California 

Map  by  Major  WIKIam  P.  Banker  USA 
Field  Checked  by  Captian  Simon  R.  Goerger  USA 
May  1998 


Figure  N.20.  V2-1  Executed  Route 


Ft.  Ord.  California 

Map  by  M  ajor  William  P.  Banker  USA 
Checked  by  Captian  Simon  R.  Goerger  USA 


Checked  fay  Captian  Simon  R.  Goerger  USA 
May  1998 


Figure  N.24.  VI -2  Executed  Route 
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14.  PARTICIPANT  NUMBER  Vl-3 


Figure  N.25.  Vl-3  Planned  Route  ; 
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16.  PARTICIPANT  NUMBER  Vl-5 


Figure  N.29.  Vl-5  Planned  Route 
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Reid  Checked  by  Captian  Simon  R.  Goerger  USA 
May  1998 


•►■Subject's  Planned  Route  t,.  i  r.i  . 

Subject's  Adjusted  Route  Plan  Figure  N.31,  V2-1  Planned  Route 


Figure  N.35.  V2-3  Planned  Route 


20.  PARTICIPANT  NUMBER  V24 
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Figure  N.38.  V2-4  Executed  Route 
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APPENDIX  O.  RAW  DATA 

1.  GENERAL  INFORMATION 

Partic^ant  data  is  referenced  by  the  participant  identification  (ID)  label  (Ml  -  Train-to- 
Standard  Map-Only  Group,  M2  -  Low  Fidelity  Map  Map-Only  Group,  VEl  -  Train-to- 
Standard  VE,  VE2  -  Low  Fidelity  Map  VE).  The  number  corresponds  to  the 
participant’s  internal  group  label.  Data  fields  that  are  left  blank  represent  information  not 
recorded  because  a  participant  did  not  undergo  the  test  or  failed  to  reach  that  point  in  the 
course. 


Vl-1 

VEl 

22-Apr-99 

11:57 

32 

M 

03 

Army 

AR 

Intermediate 

Vl-2 

VEl 

29-Apr-99 

11:48 

31 

M 

03 

Army 

AV 

Intermediate 

Ml-1 

MAPI 

23-Apr-99 

9:22 

32 

M 

03 

Marine 

SC 

Intermediate 

Ml-2 

MAPI 

30-Apr-99 

9:30 

31 

M 

03 

Army 

INF 

Expert 

Vl-3 

VEl 

7-May-99 

10:00 

32 

M 

04 

Army 

AR 

Intermediate 

Ml-3 

MAPI 

13-May-99 

12:00 

36 

M 

04 

Army 

ADA 

Expert 

Vl-4 

VEl 

14-May-99 

13:00 

32 

M 

03 

Marine 

SC 

Intermediate 

Vl-5 

VEl 

15-May-99 

10:00 

30 

M 

03 

Army 

SC 

Intermediate 

Ml-4 

MAPI 

21-May-99 

9:00 

32 

F 

03 

Army 

SC 

Intermediate 

Ml-5 

MAPI 

5-Jun-99 

8:00 

36 

M 

03 

Marine 

SC 

Expert 

V2-1 

VE2 

ll-Jun-99 

9:00 

30 

F 

03 

Marine 

SC 

Intermediate 

M2-1 

MAP2 

3-3UI-99 

6:30 

32 

M 

03 

Marine 

INF/FIN 

Intermediate 

V2-2 

VE2 

13-jul-99 

8:00 

29 

M 

03 

Marine 

SC 

Intermediate 

M2-2 

MAP2 

150uI-'99 

13:30 

31 

M 

03 

Army 

ADA 

Intermediate 

V2-3 

VE2 

16-JUI-99 

7:00 

40 

M 

04 

Marine 

SC 

Intermediate 

M2-3 

MAP2 

16-JUI-99 

12:00 

34 

M 

03 

Marine 

SC 

Expert 

V2-4 

VE2 

170UI-99 

11:00 

30 

M 

03 

Army 

AR 

Intermediate 

M2-4 

MAP2 

19-3UI-99 

13:00 

32 

M 

03 

Army 

AV 

Expert 

V2-5 

VE2 

21-3UI-99 

12:00 

35 

M 

03 

Marine 

SC 

Intermediate 

M2-5 

MAP2 

23-3UI-99 

8:00 

33 

M 

03 

Army 

MI 

Expert 
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2.  INITIAL  TESTES  AND  QUESTIONNAIRE  RESULTS 

The  initial  tests  and  questionnaires  are  in  Appendix  E.  Answers  for  the  Map  Test 
are  located  in  Appendix  E.l. 


Ml-1 

18.75 

Low 

17.5 

Intermediate 

2.80 

High 

Ml-2 

15.5 

Low 

19 

Expert 

1.93 

High 

Ml-3 

8.25 

Low 

17 

Intermediate 

2.80 

High 

Ml-4 

24.25 

High 

19.5 

Intermediate 

1.73 

High 

Ml-5 

21.5 

High 

18.5 

Expert 

1.80 

High 

M2-1 

27 

High 

19 

Intermediate 

2.87 

High 

M2-2 

42.25 

High 

18 

Expert 

2.33 

High 

M2-3 

56.25 

High 

18 

Beginner 

3.20 

High 

M2-4 

20.75 

High 

19 

Intermediate 

1.47 

High 

M2-5 

14.5 

Low 

18.5 

Intermediate 

2.33 

High 

Vl-1 

23 

High 

19.5 

Intermediate 

1.67 

High 

Vl-2 

42.25 

High 

20 

Intermediate 

2.80 

High 

Vl-3 

10.5 

Low 

19 

Intermediate 

4.33 

Low 

Vl-4 

19.5 

Low 

19.5 

Intermediate 

2.80 

High 

Vl-5 

35 

High 

19.5 

Intermediate 

3.47 

High 

V2-1 

10.75 

Low 

17 

Beginner 

3.13 

High 

V2-2 

29.25 

High 

17.5 

Intermediate 

2.93 

High 

V2-3 

31.25 

High 

19 

Intermediate 

2.27 

High 

V2-4 

14.25 

Low 

18.5 

Intermediate 

2.87 

High 

V2-5 

23 

High 

17 

Intermediate 

2.93 

High 

3.  ROUTE  ERRORS 

The  data  provided  in  this  section  consists  of  the  map  checks,  errors,  error 
distances,  and  route  leg  classifications.  The  data  appears  in  its  raw  form,  summations, 
and  normalized  form  for  each  of  the  experiment’s  twenty  participants.  The  abbreviations 
utilized  are  listed  in  Table  0.1. 


226 


Abbreviation 

Category 

C-# 

Compass  Check  -  Leg  Number 

M-# 

Map  Check  -  Leg  Number 

MC-# 

Map  and  Compass  Check  -  Leg  Number 

MCL-# 

Map  and  Compass  Check,  Location  Provided  by  Monitor  -  Leg  Number 

OB-# 

Out  of  Bounds  -  Leg  Number 

NewRt-# 

New  Route  Planned  -  Leg  Number 

Table  0.1.  Route  Errors  Abbreviation  Table 


a.  Route  Data  Summation 


Ml-l  1.60  1.60  9  8 

Ml-2  1.56 _ 1^56 _ 9 _ 9 

Ml-3 _ 1^80 _ 1.80 _ 9 _ 9 

Ml-4  1.80  1.70  9  9 


Vl-1 _ 1^20 _ ;L20 _ 9 _ 9 

Vl-2 _ L;60 _ 130 _ 9 _ 9 

Vl-3 _ 1^60 _ 1^30 _ 9  9 

Vl-4 _ 130 _ 130 _ 9 _ 9 

Vl-5  1.90  1.90  9  9 
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b.  Route  Data  Leg  SP  to  CPI 
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d.  Route  Data  Leg  CP2  to  CPS 
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e.  Route  Data  Leg  CPS  to  CP4 
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f.  Route  Data  Leg  CP4  to  CP5 
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g.  Route  Data  Leg  CPS  to  CP6 


h.  Route  Data  Leg  CPS  to  CP7 
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i.  Route  Data  Leg  CP7  to  CPS 
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j.  Route  Data  Leg  CPS  to  CP9 


0 


0 


0 


0 


0 


^SES;£sai3iSiiS!aisi*aiM.^Mi^^aa^ 

I3BI5S3gB£MgiaSEgigaEgSSM3Bg^ 

'~T~ 


Vl-1 

0 

0 

Vl-2 

2 

1009 

Vl-3 

2 

291 

Vl-4 

0 

0 

Vl-S 

0 

0 

0 

0 

0 

0 

2 

0 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

^SeaiSss^Bs^ssiE* . 


k.  Route  Data  Non  Error  Checks 


1.  Route  Data  Totals 
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m.  Leg  Difficulty  Evaluation  Banker 

Each  leg  is  evaluated  utilizing  MAJ  Banker’s  Route  Classification  (Appendix 
L.2).  “B”  stands  for  Beginner,  ‘T’  stands  or  Intermediate,  and  “A”  stands  for  Advanced. 
The  total  is  based  on  a  point  value  system  of  B  =  1, 1  =2,  and  A  =  3.  The  Average  is  the 
total  divided  by  the  number  of  legs.  0-1.50  is  an  average  course  difficulty  of  Beginner, 
1.51-2.5  is  Intermediate,  and  2.51-3.0  is  Advanced.  The  same  criteria  apply  to  the  Lisp 
leg  difficulty  evaluation. 
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4.  WHEEL  TEST  RESULTS 

a.  Wheel  Test  Results  for  Control  Point  2 


b.  wheel  Test  Results  for  Control  Point  4  and  Total  Wheel  Test  Angular  Difference 
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5.  WHITE  BOARD  RESULTS 

a.  White  Board  Normalized  Distance  Differences  from  Actual  Normalized 
Distances 
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b.  White  Board  Angles 
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V^5  I  169.522  I  132.673  |  164.489  |  123.135  |  84.315  j  112.809  |  89383  I  158.737  I  159.041 


c.  White  Board  Angles  Differences  from  Actual  Angles  and  Totals 
Su^ect  SP43  W  234  3^3  4^6  W  6,73  I  73,9  I  833P 
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APPENDIX  P.  NAVIGATION  TERMS 


The  following  navigational  terms  are  used  in  this  thesis.  This  appendix  is  included 
to  explain  the  terms  in  more  detail  than  is  allowed  in  the  thesis  body. 

HANDRAILS  -  A  linear  feature  that  runs  parallel  to  the  route  the  subject  is  following. 
This  can  be  used  to  provide  directional  cues.  For  example,  a  subject  may  choose  a  route 
which  handrails  a  creek.  As  long  as  the  subject  can  see  the  creek  to  his  side,  he  knows  he 
is  on  course. 

CONTROL  POINT  -  This  is  the  marker  which  is  what  orienteerer/student  has  to  find 
when  taking  part  in  a  land  navigation  course.  This  marker  may  be  a  flag,  post,  sign,  or 
other  device  which  marks  the  target  location. 

CHECKPOINTS  -  A  prominent  point  that  you  can  be  sure  to  identify,  both  on  the  ground 
and  on  the  map.  These  are  sometimes  known  as  reference  points.  Military  navigators 
using  dead  reckoning  are  advised  to  dead  reckon  between  interim  checkpoints  as  opposed 
to  trying  to  do  it  in  one  step. 

CATCHING  FEATURES  -  A  linear  feature  behind  or  beyond  your  intended  destination. 
Much  like  a  backstop  in  baseball,  this  is  meant  to  stop  you  and  let  you  know  that  you 
have  gone  too  far. 

NAVIGATIONAL  CORRIDORS  -  This  is  having  a  handrail  to  either  side,  preventing 
you  from  straying  too  far  to  the  left  or  right.  Combined  with  a  catching  featiure,  this 
“boxes  in”  the  destination. 

ATTACK  POINT  -  This  is  used  when  the  desired  objective  is  small  or  may  be  difiScult 
to  locate.  The  attack  point  is  an  easily  recognizable  checkpoint  near  the  objective.  You 
first  locate  the  attack  point,  then  plan  a  precise  movement  to  the  objective.  If  you  miss 
finding  the  objective,  you  merely  return  to  the  attack  point  and  try  again. 
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EXPANDED  OBJECTIVE  -  Similar  to  an  attack  point,  this  is  used  when  the  objective  is 
part  of  a  larger  terrain  feature,  say  a  hilltop.  This  simplifies  the  problem  as  you  first 
locate  the  larger  terrain  feature  (Le.,  the  hill),  then  continue  on  to  the  actual  objective. 


ROUGH  COMPASS  -  This  involves  using  a  con^ass  to  establish  a  general  direction  of 
travel  to  the  next  destination  area  or  checkpoint.  This  is  useful  if  the  next  target  area  is  a 
road  or  other  linear  feature. 

BOX  AROUND  —  This  techmque  is  used  when  following  a  dead  reckoning  azimuth  and 
a  significant  obstacle  is  encountered.  The  subject  would  turn  at  90  degrees  to  the 
direction  of  travel  and  proceed  at  a  measured  distance  (the  box  pace  count)  until  the 
obstacle  was  clear  to  his  side.  The  subject  then  continues  parallel  to  the  line  of  original 
travel  resuming  the  original  pace  count  until  the  obstacle  was  cleared  again.  Now  the 
subject  turns  90  degrees  in  the  opposite  direction,  repeats  the  original  measured  distance 
(the  box  pace  count),  and  then  returns  to  the  original  direction  of  travel  at  the  pace  count 
they  had  at  the  end  of  the  parallel  leg.  Difficulties  arise  when  the  obstacle  is  not  uniform 
in  size.  Having  to  maintain  two  different  pace  counts  can  also  be  difficult  for  some 
individuals. 
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